2024-09-21T22:56:00.321 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T22:56:00.338 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T22:56:00.414 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914506' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1478 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 24/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEc63V0g2edqYC8QUmQeCrgtINQVX4mxm/M5jzALBN1ZatwWDIQTkVcaqybmtAKY7ScUgqIfdVqFt8ZccNgdmoA= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNvwl4cIDU8XsIRA5ZHXmUbzJCfcrQIsMg/t/Id+xBqgwaHnqOHyOYrNc7mqWYLJN/O+uZ24F2ignLoqiiHV1i0= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T22:56:00.414 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-21T22:56:00.415 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-21T22:56:00.415 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T22:56:00.417 INFO:teuthology.task.internal:Checking packages... 2024-09-21T22:56:00.437 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-21T22:56:00.437 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T22:56:00.437 INFO:teuthology.packaging:ref: None 2024-09-21T22:56:00.437 INFO:teuthology.packaging:tag: None 2024-09-21T22:56:00.437 INFO:teuthology.packaging:branch: squid 2024-09-21T22:56:00.437 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T22:56:00.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-21T22:56:00.641 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T22:56:00.642 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T22:56:00.649 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T22:56:00.649 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T22:56:00.655 INFO:teuthology.task.internal:Saving configuration 2024-09-21T22:56:00.668 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T22:56:00.674 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T22:56:00.696 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 22:50:14.094895', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN3izxXU4O94qxh4Qijx4Fx6cK5+oGWOOSGoKl0P8gb2'} 2024-09-21T22:56:00.715 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 22:50:14.093103', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMR4OlTn6i0+rUhSu8Nq6ehIbcoA1fFy5iwyhqqX9J5h'} 2024-09-21T22:56:00.715 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T22:56:00.721 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-21T22:56:00.721 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-21T22:56:00.721 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T22:56:00.767 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f806fcff880>, signals=[15]) 2024-09-21T22:56:00.767 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T22:56:00.775 INFO:teuthology.task.internal:Opening connections... 2024-09-21T22:56:00.775 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T22:56:00.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:56:00.856 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T22:56:00.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:56:00.932 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T22:56:00.938 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-09-21T22:56:00.943 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-09-21T22:56:00.943 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-09-21T22:56:00.989 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Ubuntu" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="22.04" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_CODENAME=jammy 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:ID=ubuntu 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE=debian 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T22:56:00.990 INFO:teuthology.orchestra.run.smithi150.stdout:UBUNTU_CODENAME=jammy 2024-09-21T22:56:00.991 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-09-21T22:56:01.008 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-09-21T22:56:01.014 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-09-21T22:56:01.014 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-09-21T22:56:01.058 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T22:56:01.058 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Ubuntu" 2024-09-21T22:56:01.058 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="22.04" 2024-09-21T22:56:01.058 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T22:56:01.058 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_CODENAME=jammy 2024-09-21T22:56:01.058 INFO:teuthology.orchestra.run.smithi175.stdout:ID=ubuntu 2024-09-21T22:56:01.059 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE=debian 2024-09-21T22:56:01.059 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T22:56:01.059 INFO:teuthology.orchestra.run.smithi175.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T22:56:01.059 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T22:56:01.059 INFO:teuthology.orchestra.run.smithi175.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T22:56:01.059 INFO:teuthology.orchestra.run.smithi175.stdout:UBUNTU_CODENAME=jammy 2024-09-21T22:56:01.059 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-09-21T22:56:01.075 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T22:56:01.083 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T22:56:01.088 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T22:56:01.088 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-09-21T22:56:01.091 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-09-21T22:56:01.106 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T22:56:01.111 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T22:56:01.111 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-09-21T22:56:01.139 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-09-21T22:56:01.225 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T22:56:01.280 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T22:56:01.293 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T22:56:01.293 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T22:56:01.293 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T22:56:01.294 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T22:56:01.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:56:01.298 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-21T22:56:01.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T22:56:01.299 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-21T22:56:01.346 INFO:teuthology.orchestra.run.smithi150.stdout:5.15.0-56-generic 2024-09-21T22:56:01.346 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.15.0-56-generic 2024-09-21T22:56:01.346 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2024-09-21T22:56:01.347 INFO:teuthology.orchestra.run.smithi175.stdout:5.15.0-56-generic 2024-09-21T22:56:01.347 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.15.0-56-generic 2024-09-21T22:56:01.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get clean 2024-09-21T22:56:01.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get update 2024-09-21T22:56:01.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2024-09-21T22:56:01.642 INFO:teuthology.orchestra.run.smithi175.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T22:56:01.659 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:56:01.659 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T22:56:01.660 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T22:56:01.719 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T22:56:01.753 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:56:01.837 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T22:56:01.854 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T22:56:01.897 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T22:56:01.955 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T22:56:01.959 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T22:56:01.962 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T22:56:01.962 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T22:56:01.971 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T22:56:01.981 INFO:teuthology.orchestra.run.smithi175.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T22:56:01.982 INFO:teuthology.orchestra.run.smithi175.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T22:56:01.985 INFO:teuthology.orchestra.run.smithi175.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T22:56:01.985 INFO:teuthology.orchestra.run.smithi175.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T22:56:01.990 INFO:teuthology.orchestra.run.smithi175.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T22:56:01.996 INFO:teuthology.orchestra.run.smithi175.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T22:56:01.998 INFO:teuthology.orchestra.run.smithi175.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T22:56:01.998 INFO:teuthology.orchestra.run.smithi175.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T22:56:01.998 INFO:teuthology.orchestra.run.smithi175.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T22:56:02.012 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T22:56:02.012 INFO:teuthology.orchestra.run.smithi175.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T22:56:02.017 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T22:56:02.019 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T22:56:02.020 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T22:56:02.028 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T22:56:02.032 INFO:teuthology.orchestra.run.smithi175.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T22:56:02.047 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T22:56:02.051 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T22:56:02.051 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T22:56:02.055 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T22:56:02.058 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T22:56:02.062 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T22:56:02.063 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T22:56:02.113 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T22:56:02.113 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T22:56:02.113 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T22:56:02.114 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T22:56:02.114 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T22:56:02.114 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T22:56:02.115 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T22:56:02.115 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T22:56:02.155 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T22:56:02.161 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T22:56:02.161 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T22:56:02.170 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T22:56:02.180 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T22:56:02.183 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T22:56:02.183 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T22:56:02.184 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T22:56:02.184 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T22:56:02.184 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T22:56:02.189 INFO:teuthology.orchestra.run.smithi175.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T22:56:02.196 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T22:56:02.197 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T22:56:02.198 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T22:56:02.210 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T22:56:02.210 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T22:56:02.210 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T22:56:02.210 INFO:teuthology.orchestra.run.smithi150.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T22:56:02.211 INFO:teuthology.orchestra.run.smithi150.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T22:56:02.327 INFO:teuthology.orchestra.run.smithi175.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T22:56:02.647 INFO:teuthology.orchestra.run.smithi175.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T22:56:02.681 INFO:teuthology.orchestra.run.smithi175.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T22:56:02.696 INFO:teuthology.orchestra.run.smithi175.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T22:56:02.697 INFO:teuthology.orchestra.run.smithi175.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T22:56:02.766 INFO:teuthology.orchestra.run.smithi175.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T22:56:02.767 INFO:teuthology.orchestra.run.smithi175.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T22:56:02.783 INFO:teuthology.orchestra.run.smithi175.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T22:56:02.783 INFO:teuthology.orchestra.run.smithi175.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T22:56:02.806 INFO:teuthology.orchestra.run.smithi175.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T22:56:02.835 INFO:teuthology.orchestra.run.smithi175.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T22:56:02.843 INFO:teuthology.orchestra.run.smithi175.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T22:56:02.844 INFO:teuthology.orchestra.run.smithi175.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T22:56:02.845 INFO:teuthology.orchestra.run.smithi175.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T22:56:02.892 INFO:teuthology.orchestra.run.smithi175.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T22:56:02.892 INFO:teuthology.orchestra.run.smithi175.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T22:56:02.893 INFO:teuthology.orchestra.run.smithi175.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T22:56:02.895 INFO:teuthology.orchestra.run.smithi175.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T22:56:02.896 INFO:teuthology.orchestra.run.smithi175.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T22:56:02.973 INFO:teuthology.orchestra.run.smithi175.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T22:56:02.974 INFO:teuthology.orchestra.run.smithi175.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T22:56:02.974 INFO:teuthology.orchestra.run.smithi175.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T22:56:02.975 INFO:teuthology.orchestra.run.smithi175.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T22:56:02.976 INFO:teuthology.orchestra.run.smithi175.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T22:56:08.250 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 16.3 MB in 2s (7,392 kB/s) 2024-09-21T22:56:08.417 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 16.3 MB in 2s (7,312 kB/s) 2024-09-21T22:56:09.643 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-09-21T22:56:09.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:56:09.723 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-09-21T22:56:09.922 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-21T22:56:09.934 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2024-09-21T22:56:09.935 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2024-09-21T22:56:09.941 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:56:10.008 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-21T22:56:10.109 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:56:10.109 INFO:teuthology.orchestra.run.smithi175.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:56:10.109 INFO:teuthology.orchestra.run.smithi175.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:56:10.110 INFO:teuthology.orchestra.run.smithi175.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:56:10.110 INFO:teuthology.orchestra.run.smithi175.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T22:56:10.110 INFO:teuthology.orchestra.run.smithi175.stdout: python2.7-minimal 2024-09-21T22:56:10.110 INFO:teuthology.orchestra.run.smithi175.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:56:10.111 INFO:teuthology.orchestra.run.smithi175.stdout:The following additional packages will be installed: 2024-09-21T22:56:10.111 INFO:teuthology.orchestra.run.smithi175.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:56:10.111 INFO:teuthology.orchestra.run.smithi175.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T22:56:10.112 INFO:teuthology.orchestra.run.smithi175.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T22:56:10.113 INFO:teuthology.orchestra.run.smithi175.stdout:Suggested packages: 2024-09-21T22:56:10.113 INFO:teuthology.orchestra.run.smithi175.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T22:56:10.113 INFO:teuthology.orchestra.run.smithi175.stdout:Recommended packages: 2024-09-21T22:56:10.113 INFO:teuthology.orchestra.run.smithi175.stdout: thermald 2024-09-21T22:56:10.160 INFO:teuthology.orchestra.run.smithi175.stdout:The following NEW packages will be installed: 2024-09-21T22:56:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:56:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T22:56:10.160 INFO:teuthology.orchestra.run.smithi175.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T22:56:10.161 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages will be upgraded: 2024-09-21T22:56:10.162 INFO:teuthology.orchestra.run.smithi175.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T22:56:10.220 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2024-09-21T22:56:10.220 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2024-09-21T22:56:10.341 INFO:teuthology.orchestra.run.smithi175.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:56:10.341 INFO:teuthology.orchestra.run.smithi175.stdout:Need to get 113 MB of archives. 2024-09-21T22:56:10.341 INFO:teuthology.orchestra.run.smithi175.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T22:56:10.341 INFO:teuthology.orchestra.run.smithi175.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T22:56:10.414 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:56:10.414 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:56:10.414 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:56:10.414 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:56:10.414 INFO:teuthology.orchestra.run.smithi150.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T22:56:10.415 INFO:teuthology.orchestra.run.smithi150.stdout: python2.7-minimal 2024-09-21T22:56:10.415 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:56:10.416 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2024-09-21T22:56:10.416 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:56:10.416 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T22:56:10.416 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T22:56:10.418 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2024-09-21T22:56:10.418 INFO:teuthology.orchestra.run.smithi150.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T22:56:10.418 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2024-09-21T22:56:10.418 INFO:teuthology.orchestra.run.smithi150.stdout: thermald 2024-09-21T22:56:10.464 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2024-09-21T22:56:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T22:56:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T22:56:10.464 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T22:56:10.466 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages will be upgraded: 2024-09-21T22:56:10.466 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T22:56:10.518 INFO:teuthology.orchestra.run.smithi150.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:56:10.518 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 113 MB of archives. 2024-09-21T22:56:10.518 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T22:56:10.518 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T22:56:10.938 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T22:56:11.037 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T22:56:11.756 INFO:teuthology.orchestra.run.smithi175.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T22:56:11.847 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T22:56:11.847 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T22:56:11.847 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T22:56:11.975 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T22:56:11.997 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T22:56:12.106 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T22:56:12.410 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 113 MB in 2s (74.4 MB/s) 2024-09-21T22:56:12.586 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T22:56:14.380 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T22:56:14.437 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T22:56:14.441 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T22:56:14.651 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T22:56:14.655 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:14.775 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:14.799 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T22:56:14.884 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T22:56:15.294 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 113 MB in 5s (24.0 MB/s) 2024-09-21T22:56:15.476 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T22:56:16.902 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T22:56:16.923 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:17.011 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:17.387 INFO:teuthology.orchestra.run.smithi175.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T22:56:17.391 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:17.473 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:17.488 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T22:56:17.509 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:17.546 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:19.482 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T22:56:19.503 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:19.573 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:20.060 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T22:56:20.081 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:20.110 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:24.642 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:56:24.750 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:56:25.049 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:56:25.178 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:56:25.481 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T22:56:25.504 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T22:56:25.573 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:56:26.776 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:56:26.919 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:56:27.236 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:56:27.339 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:56:27.642 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T22:56:27.665 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T22:56:27.709 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:56:31.486 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T22:56:31.523 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:31.560 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:33.592 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T22:56:33.630 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T22:56:33.675 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:34.333 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:56:34.456 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:56:34.831 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:56:34.961 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:35.112 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T22:56:35.233 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:36.257 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T22:56:36.379 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T22:56:36.535 INFO:teuthology.orchestra.run.smithi150.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:56:36.535 INFO:teuthology.orchestra.run.smithi150.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T22:56:36.536 INFO:teuthology.orchestra.run.smithi150.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:56:36.536 INFO:teuthology.orchestra.run.smithi150.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T22:56:36.692 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:36.713 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T22:56:36.834 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:36.960 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T22:56:37.072 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:38.494 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T22:56:38.620 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T22:56:38.737 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:39.197 INFO:teuthology.orchestra.run.smithi175.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:56:39.198 INFO:teuthology.orchestra.run.smithi175.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T22:56:39.198 INFO:teuthology.orchestra.run.smithi175.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:56:39.198 INFO:teuthology.orchestra.run.smithi175.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T22:56:39.358 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:40.089 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:40.150 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T22:56:40.150 INFO:teuthology.orchestra.run.smithi150.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:56:40.751 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T22:56:40.852 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T22:56:40.961 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:42.371 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T22:56:42.437 INFO:teuthology.orchestra.run.smithi175.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T22:56:42.437 INFO:teuthology.orchestra.run.smithi175.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:56:54.657 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T22:56:54.657 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub' 2024-09-21T22:56:54.681 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:56:54.705 INFO:teuthology.orchestra.run.smithi150.stdout:Generating grub configuration file ... 2024-09-21T22:56:54.985 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:56:55.012 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:56:55.341 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:56:55.346 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:56:55.415 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:56:55.420 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:56:55.587 INFO:teuthology.orchestra.run.smithi150.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:56:55.587 INFO:teuthology.orchestra.run.smithi150.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:56:55.587 INFO:teuthology.orchestra.run.smithi150.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:56:55.626 INFO:teuthology.orchestra.run.smithi150.stdout:done 2024-09-21T22:56:56.150 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2024-09-21T22:56:56.165 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 21 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.15.0.122.122 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2024-09-21T22:56:56.166 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2024-09-21T22:56:56.167 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:56:56.167 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2024-09-21T22:56:56.167 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T22:56:56.167 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T22:56:56.167 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T22:56:56.167 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T22:56:56.167 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T22:56:56.167 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2024-09-21T22:56:56.324 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2024-09-21T22:56:56.440 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:56:56.440 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T22:56:56.449 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T22:56:56.466 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T22:56:56.750 INFO:teuthology.orchestra.run.smithi175.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T22:56:56.751 INFO:teuthology.orchestra.run.smithi175.stdout:Sourcing file `/etc/default/grub' 2024-09-21T22:56:56.772 INFO:teuthology.orchestra.run.smithi175.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:56:56.794 INFO:teuthology.orchestra.run.smithi175.stdout:Generating grub configuration file ... 2024-09-21T22:56:57.102 INFO:teuthology.orchestra.run.smithi175.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:56:57.115 INFO:teuthology.orchestra.run.smithi175.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:56:57.465 INFO:teuthology.orchestra.run.smithi175.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:56:57.470 INFO:teuthology.orchestra.run.smithi175.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:56:57.546 INFO:teuthology.orchestra.run.smithi175.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:56:57.552 INFO:teuthology.orchestra.run.smithi175.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:56:57.702 INFO:teuthology.orchestra.run.smithi175.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:56:57.703 INFO:teuthology.orchestra.run.smithi175.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:56:57.703 INFO:teuthology.orchestra.run.smithi175.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:56:57.744 INFO:teuthology.orchestra.run.smithi175.stdout:done 2024-09-21T22:56:57.813 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-21T22:56:57.832 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:56:57.886 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-21T22:56:58.082 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2024-09-21T22:56:58.083 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2024-09-21T22:56:58.232 DEBUG:teuthology.orchestra.run.smithi175:> dpkg -s linux-image-generic 2024-09-21T22:56:58.253 INFO:teuthology.orchestra.run.smithi150.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T22:56:58.253 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:56:58.253 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:56:58.253 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:56:58.253 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:56:58.253 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T22:56:58.253 INFO:teuthology.orchestra.run.smithi150.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T22:56:58.254 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Package: linux-image-generic 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Status: install ok installed 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Priority: optional 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Section: kernel 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Installed-Size: 21 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: amd64 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Source: linux-meta 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Version: 5.15.0.122.122 2024-09-21T22:56:58.259 INFO:teuthology.orchestra.run.smithi175.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:56:58.260 INFO:teuthology.orchestra.run.smithi175.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:56:58.260 INFO:teuthology.orchestra.run.smithi175.stdout:Recommends: thermald 2024-09-21T22:56:58.260 INFO:teuthology.orchestra.run.smithi175.stdout:Description: Generic Linux kernel image 2024-09-21T22:56:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:56:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: available. 2024-09-21T22:56:58.260 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T22:56:58.260 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T22:56:58.260 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T22:56:58.260 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T22:56:58.260 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T22:56:58.261 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get clean 2024-09-21T22:56:58.295 INFO:teuthology.orchestra.run.smithi150.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:56:58.297 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2024-09-21T22:56:58.355 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2024-09-21T22:56:58.355 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2024-09-21T22:56:58.355 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2024-09-21T22:56:58.355 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 21 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.15.0.122.122 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:56:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2024-09-21T22:56:58.357 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-21T22:56:58.404 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.I3s4FiLh6L 2024-09-21T22:56:58.405 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub/grub.cfg /tmp/tmp.I3s4FiLh6L 2024-09-21T22:56:58.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get update 2024-09-21T22:56:58.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.I3s4FiLh6L 2024-09-21T22:56:58.575 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T22:56:58.586 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T22:56:58.589 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.I3s4FiLh6L is 10KB 2024-09-21T22:56:58.601 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.I3s4FiLh6L 2024-09-21T22:56:58.603 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T22:56:58.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T22:56:58.661 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T22:56:58.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T22:56:58.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T22:56:58.698 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T22:56:58.728 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T22:56:58.728 DEBUG:teuthology.orchestra.run.smithi150:> sudo update-grub 2024-09-21T22:57:00.016 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-09-21T22:57:00.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T22:57:00.087 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub' 2024-09-21T22:57:00.089 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:57:00.092 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-09-21T22:57:00.098 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-09-21T22:57:00.310 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2024-09-21T22:57:00.311 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2024-09-21T22:57:00.328 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:57:00.334 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:57:00.502 INFO:teuthology.orchestra.run.smithi175.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T22:57:00.502 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T22:57:00.502 INFO:teuthology.orchestra.run.smithi175.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T22:57:00.502 INFO:teuthology.orchestra.run.smithi175.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T22:57:00.502 INFO:teuthology.orchestra.run.smithi175.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T22:57:00.503 INFO:teuthology.orchestra.run.smithi175.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T22:57:00.503 INFO:teuthology.orchestra.run.smithi175.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T22:57:00.503 INFO:teuthology.orchestra.run.smithi175.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T22:57:00.543 INFO:teuthology.orchestra.run.smithi175.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T22:57:00.545 DEBUG:teuthology.orchestra.run.smithi175:> dpkg -s linux-image-generic 2024-09-21T22:57:00.550 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:57:00.555 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Package: linux-image-generic 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Status: install ok installed 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Priority: optional 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Section: kernel 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Installed-Size: 21 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: amd64 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Source: linux-meta 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Version: 5.15.0.122.122 2024-09-21T22:57:00.560 INFO:teuthology.orchestra.run.smithi175.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T22:57:00.561 INFO:teuthology.orchestra.run.smithi175.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T22:57:00.561 INFO:teuthology.orchestra.run.smithi175.stdout:Recommends: thermald 2024-09-21T22:57:00.561 INFO:teuthology.orchestra.run.smithi175.stdout:Description: Generic Linux kernel image 2024-09-21T22:57:00.561 INFO:teuthology.orchestra.run.smithi175.stdout: This package will always depend on the latest generic kernel image 2024-09-21T22:57:00.561 INFO:teuthology.orchestra.run.smithi175.stdout: available. 2024-09-21T22:57:00.561 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-21T22:57:00.597 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:57:00.602 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:57:00.605 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.mY9XFrryE4 2024-09-21T22:57:00.606 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub/grub.cfg /tmp/tmp.mY9XFrryE4 2024-09-21T22:57:00.661 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.mY9XFrryE4 2024-09-21T22:57:00.732 INFO:teuthology.orchestra.run.smithi150.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:57:00.732 INFO:teuthology.orchestra.run.smithi150.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:57:00.732 INFO:teuthology.orchestra.run.smithi150.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:57:00.749 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-09-21T22:57:00.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-09-21T22:57:00.772 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.mY9XFrryE4 is 10KB 2024-09-21T22:57:00.787 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.mY9XFrryE4 2024-09-21T22:57:00.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T22:57:00.846 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T22:57:00.846 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T22:57:00.846 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T22:57:00.914 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T22:57:00.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo update-grub 2024-09-21T22:57:02.182 INFO:teuthology.orchestra.run.smithi175.stderr:Sourcing file `/etc/default/grub' 2024-09-21T22:57:02.183 INFO:teuthology.orchestra.run.smithi175.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T22:57:02.186 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-09-21T22:57:02.352 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T22:57:02.359 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T22:57:02.629 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T22:57:02.634 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T22:57:02.672 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T22:57:02.677 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T22:57:02.788 INFO:teuthology.orchestra.run.smithi175.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T22:57:02.788 INFO:teuthology.orchestra.run.smithi175.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T22:57:02.788 INFO:teuthology.orchestra.run.smithi175.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T22:57:02.813 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-09-21T22:57:02.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-09-21T22:57:30.778 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T22:57:30.778 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T22:57:30.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:57:32.832 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T22:57:32.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T22:57:32.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:57:51.343 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-09-21T22:58:00.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T22:58:00.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:58:03.407 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-09-21T22:58:05.423 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-21T22:58:14.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T22:58:14.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:58:15.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T22:58:15.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:58:33.839 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-09-21T22:58:48.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T22:58:48.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:58:49.144 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-21T22:58:49.719 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T22:58:49.719 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-21T22:58:49.719 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-21T22:58:49.766 INFO:teuthology.orchestra.run.smithi175.stdout:5.15.0-122-generic 2024-09-21T22:58:49.766 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T22:58:49.766 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T22:58:49.766 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T22:58:50.768 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T22:58:50.768 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T22:58:50.845 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-09-21T22:58:50.869 DEBUG:teuthology.parallel:result is None 2024-09-21T22:59:14.434 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T22:59:26.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T22:59:26.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T22:59:26.677 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-21T22:59:27.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T22:59:27.270 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-21T22:59:27.270 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-21T22:59:27.315 INFO:teuthology.orchestra.run.smithi150.stdout:5.15.0-122-generic 2024-09-21T22:59:27.315 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T22:59:27.315 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T22:59:27.315 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T22:59:28.315 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T22:59:28.316 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T22:59:28.699 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-09-21T22:59:28.729 DEBUG:teuthology.parallel:result is None 2024-09-21T22:59:28.729 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T22:59:28.737 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T22:59:28.737 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T22:59:28.739 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T22:59:28.745 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T22:59:28.751 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T22:59:28.760 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T22:59:28.760 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T22:59:28.788 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T22:59:29.012 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T22:59:29.018 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T22:59:29.018 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T22:59:29.021 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T22:59:29.045 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:59:29.047 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:59:29.053 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:59:29.056 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T22:59:29.058 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T22:59:29.065 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T22:59:29.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T22:59:29.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T22:59:29.114 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T22:59:29.123 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T22:59:29.123 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T22:59:29.157 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T22:59:29.162 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T22:59:29.207 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T22:59:29.255 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T22:59:29.256 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T22:59:29.316 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T22:59:29.322 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T22:59:29.370 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T22:59:29.371 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T22:59:29.426 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-09-21T22:59:29.429 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-09-21T22:59:29.507 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T22:59:29.514 INFO:teuthology.task.internal:Starting timer... 2024-09-21T22:59:29.514 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T22:59:29.524 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T22:59:29.533 DEBUG:teuthology.task.selinux:Excluding smithi150: OS 'ubuntu' does not support SELinux 2024-09-21T22:59:29.533 DEBUG:teuthology.task.selinux:Excluding smithi175: OS 'ubuntu' does not support SELinux 2024-09-21T22:59:29.533 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T22:59:29.534 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T22:59:29.534 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T22:59:29.534 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T22:59:29.542 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T22:59:29.552 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T22:59:29.553 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi150.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T23:06:51.009 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-09-21T23:06:51.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T23:06:51.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:06:51.093 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-21T23:06:51.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-21T23:06:51.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T23:06:51.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:06:51.249 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-21T23:06:51.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T23:06:51.322 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T23:06:51.334 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T23:06:51.335 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:06:51.335 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:06:51.337 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:06:51.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:06:51.363 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:06:51.363 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Command line: ntpd -gq 2024-09-21T23:06:51.363 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: ---------------------------------------------------- 2024-09-21T23:06:51.363 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:06:51.363 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:06:51.364 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: corporation. Support and training for ntp-4 are 2024-09-21T23:06:51.364 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: available at https://www.nwtime.org/support 2024-09-21T23:06:51.364 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: ---------------------------------------------------- 2024-09-21T23:06:51.365 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: proto: precision = 0.100 usec (-23) 2024-09-21T23:06:51.365 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: basedate set to 2022-02-04 2024-09-21T23:06:51.365 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:06:51.365 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:06:51.365 INFO:teuthology.orchestra.run.smithi150.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:06:51.365 INFO:teuthology.orchestra.run.smithi150.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:06:51.366 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:06:51.366 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:06:51.366 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:06:51.366 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:06:51.367 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Listen normally on 3 enp3s0f1 172.21.15.150:123 2024-09-21T23:06:51.367 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Listen normally on 4 lo [::1]:123 2024-09-21T23:06:51.367 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e35%5]:123 2024-09-21T23:06:51.367 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:51 ntpd[18460]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:06:51.391 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:06:51.391 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Command line: ntpd -gq 2024-09-21T23:06:51.391 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: ---------------------------------------------------- 2024-09-21T23:06:51.391 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:06:51.391 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:06:51.391 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: corporation. Support and training for ntp-4 are 2024-09-21T23:06:51.392 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: available at https://www.nwtime.org/support 2024-09-21T23:06:51.392 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: ---------------------------------------------------- 2024-09-21T23:06:51.392 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: proto: precision = 0.044 usec (-24) 2024-09-21T23:06:51.392 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: basedate set to 2022-02-04 2024-09-21T23:06:51.392 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:06:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:06:51.393 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:06:51.393 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:06:51.393 INFO:teuthology.orchestra.run.smithi175.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:06:51.394 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:06:51.394 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:06:51.394 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:06:51.394 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Listen normally on 3 enp3s0f1 172.21.15.175:123 2024-09-21T23:06:51.394 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Listen normally on 4 lo [::1]:123 2024-09-21T23:06:51.394 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:71f1%5]:123 2024-09-21T23:06:51.395 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:51 ntpd[18515]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:06:52.391 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:52 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:52.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:52 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:52.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:52 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:52.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:52 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:52.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:52 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:52.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:52 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:53.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:53 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:53.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:53 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:53.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:53 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:53.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:53 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:53.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:53 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:53.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:53 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:54.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:54.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:54.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:54.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:54 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:54.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:54 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:54.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:54 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:54.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:54 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:54.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:54 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:54.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:54 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:54.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:54 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:55.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:55 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:55.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:55 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:55.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:55 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:55.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:55 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:55.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:55 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:55.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:55 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:55.391 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:55 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:55.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:55 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:55.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:55 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:55.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:55 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:55.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:55 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:55.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:55 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:56.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:56.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:56 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:56.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:56.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:56 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:57.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:57 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:57.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:57 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:57.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:57 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:57.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:57 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:57.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:57 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:57.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:57 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:57.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:57 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:57.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:57 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:57.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:57 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:57.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:57 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:57.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:57 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:57.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:57 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:58.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:58.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:58.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:58.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:58 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:58.391 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stdout:21 Sep 23:06:58 ntpd[18515]: ntpd: time slew +0.001071 s 2024-09-21T23:06:58.392 INFO:teuthology.orchestra.run.smithi175.stdout:ntpd: time slew +0.001071s 2024-09-21T23:06:58.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:06:58.393 INFO:teuthology.orchestra.run.smithi175.stderr:21 Sep 23:06:58 ntpd[18515]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:06:58.448 INFO:teuthology.orchestra.run.smithi175.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:06:58.448 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================== 2024-09-21T23:06:58.448 INFO:teuthology.orchestra.run.smithi175.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:58.448 INFO:teuthology.orchestra.run.smithi175.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:58.448 INFO:teuthology.orchestra.run.smithi175.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:58.448 INFO:teuthology.orchestra.run.smithi175.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stdout:21 Sep 23:06:59 ntpd[18460]: ntpd: time slew -0.000548 s 2024-09-21T23:06:59.364 INFO:teuthology.orchestra.run.smithi150.stdout:ntpd: time slew -0.000548s 2024-09-21T23:06:59.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:06:59.365 INFO:teuthology.orchestra.run.smithi150.stderr:21 Sep 23:06:59 ntpd[18460]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:06:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:06:59.505 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2024-09-21T23:06:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:06:59.506 INFO:teuthology.run_tasks:Running task install... 2024-09-21T23:06:59.514 DEBUG:teuthology.task.install:project ceph 2024-09-21T23:06:59.517 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T23:06:59.517 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-21T23:06:59.517 INFO:teuthology.task.install:Using flavor: default 2024-09-21T23:06:59.523 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T23:06:59.523 INFO:teuthology.task.install:extra packages: [] 2024-09-21T23:06:59.524 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-key list | grep Ceph 2024-09-21T23:06:59.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-key list | grep Ceph 2024-09-21T23:06:59.588 INFO:teuthology.orchestra.run.smithi175.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:06:59.615 INFO:teuthology.orchestra.run.smithi175.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:06:59.616 INFO:teuthology.orchestra.run.smithi175.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:06:59.616 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T23:06:59.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:06:59.621 INFO:teuthology.orchestra.run.smithi150.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:06:59.652 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:06:59.652 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:06:59.653 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T23:06:59.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:06:59.765 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:06:59.765 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:06:59.789 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:06:59.789 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:06:59.881 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:06:59.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:06:59.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get update 2024-09-21T23:06:59.897 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:06:59.897 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:06:59.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2024-09-21T23:07:00.060 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:07:00.076 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:07:00.087 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:07:00.104 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:07:00.114 INFO:teuthology.orchestra.run.smithi175.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:07:00.135 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:07:00.138 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:07:00.156 INFO:teuthology.orchestra.run.smithi175.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:07:00.159 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:07:00.177 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:07:00.180 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:07:00.182 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:07:00.199 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:07:00.202 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:07:00.203 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:07:00.228 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:07:00.264 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:07:00.346 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:07:00.407 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 28.9 kB in 0s (91.1 kB/s) 2024-09-21T23:07:00.485 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 28.9 kB in 0s (70.8 kB/s) 2024-09-21T23:07:01.398 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-21T23:07:01.415 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:07:01.467 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-21T23:07:01.486 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-09-21T23:07:01.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:07:01.565 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-09-21T23:07:01.654 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2024-09-21T23:07:01.655 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2024-09-21T23:07:01.774 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2024-09-21T23:07:01.775 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2024-09-21T23:07:01.841 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:07:01.841 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:07:01.841 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:07:01.842 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:07:01.843 INFO:teuthology.orchestra.run.smithi150.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:07:01.844 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:07:01.844 INFO:teuthology.orchestra.run.smithi150.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:07:01.845 INFO:teuthology.orchestra.run.smithi150.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet zip 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:07:01.847 INFO:teuthology.orchestra.run.smithi150.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:07:01.847 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2024-09-21T23:07:01.847 INFO:teuthology.orchestra.run.smithi150.stdout: btrfs-tools 2024-09-21T23:07:01.950 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2024-09-21T23:07:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:07:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:07:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:07:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:07:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:07:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:07:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:07:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:07:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:07:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:07:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:07:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-21T23:07:01.974 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:07:01.974 INFO:teuthology.orchestra.run.smithi175.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:07:01.974 INFO:teuthology.orchestra.run.smithi175.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:07:01.974 INFO:teuthology.orchestra.run.smithi175.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:07:01.974 INFO:teuthology.orchestra.run.smithi175.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:07:01.974 INFO:teuthology.orchestra.run.smithi175.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:07:01.974 INFO:teuthology.orchestra.run.smithi175.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout:The following additional packages will be installed: 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:07:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:07:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet zip 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout:Suggested packages: 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:07:01.978 INFO:teuthology.orchestra.run.smithi175.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:07:01.979 INFO:teuthology.orchestra.run.smithi175.stdout:Recommended packages: 2024-09-21T23:07:01.979 INFO:teuthology.orchestra.run.smithi175.stdout: btrfs-tools 2024-09-21T23:07:02.011 INFO:teuthology.orchestra.run.smithi150.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:07:02.011 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 181 MB of archives. 2024-09-21T23:07:02.011 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:07:02.011 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:07:02.080 INFO:teuthology.orchestra.run.smithi175.stdout:The following NEW packages will be installed: 2024-09-21T23:07:02.080 INFO:teuthology.orchestra.run.smithi175.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:07:02.080 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:07:02.080 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:07:02.082 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:07:02.083 INFO:teuthology.orchestra.run.smithi175.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-21T23:07:02.093 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:07:02.099 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:07:02.101 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:07:02.118 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:07:02.142 INFO:teuthology.orchestra.run.smithi175.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:07:02.142 INFO:teuthology.orchestra.run.smithi175.stdout:Need to get 181 MB of archives. 2024-09-21T23:07:02.142 INFO:teuthology.orchestra.run.smithi175.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:07:02.142 INFO:teuthology.orchestra.run.smithi175.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:07:02.177 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:07:02.181 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:07:02.194 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:07:02.198 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:07:02.198 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:07:02.199 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:07:02.208 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:07:02.210 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:07:02.212 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:07:02.213 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:07:02.216 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:07:02.218 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:07:02.225 INFO:teuthology.orchestra.run.smithi175.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:07:02.229 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:07:02.231 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:07:02.232 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:07:02.233 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:07:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:07:02.234 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:07:02.235 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:07:02.235 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:07:02.236 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:07:02.236 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:07:02.249 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:07:02.250 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:07:02.250 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:07:02.268 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:07:02.270 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:07:02.271 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:07:02.271 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:07:02.271 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:07:02.272 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:07:02.273 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:07:02.273 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:07:02.278 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:07:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:07:02.285 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:07:02.302 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:07:02.307 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:07:02.307 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:07:02.307 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:07:02.308 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:07:02.308 INFO:teuthology.orchestra.run.smithi150.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:07:02.310 INFO:teuthology.orchestra.run.smithi150.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:07:02.312 INFO:teuthology.orchestra.run.smithi150.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:07:02.312 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:07:02.312 INFO:teuthology.orchestra.run.smithi150.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:07:02.319 INFO:teuthology.orchestra.run.smithi150.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:07:02.320 INFO:teuthology.orchestra.run.smithi150.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:07:02.324 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:07:02.328 INFO:teuthology.orchestra.run.smithi175.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:07:02.329 INFO:teuthology.orchestra.run.smithi175.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:07:02.330 INFO:teuthology.orchestra.run.smithi175.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:07:02.335 INFO:teuthology.orchestra.run.smithi150.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:07:02.336 INFO:teuthology.orchestra.run.smithi150.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:07:02.337 INFO:teuthology.orchestra.run.smithi150.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:07:02.337 INFO:teuthology.orchestra.run.smithi150.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:07:02.338 INFO:teuthology.orchestra.run.smithi175.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:07:02.340 INFO:teuthology.orchestra.run.smithi150.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:07:02.340 INFO:teuthology.orchestra.run.smithi175.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:07:02.343 INFO:teuthology.orchestra.run.smithi175.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:07:02.344 INFO:teuthology.orchestra.run.smithi175.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:07:02.346 INFO:teuthology.orchestra.run.smithi175.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:07:02.348 INFO:teuthology.orchestra.run.smithi175.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:07:02.360 INFO:teuthology.orchestra.run.smithi150.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:07:02.363 INFO:teuthology.orchestra.run.smithi175.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:07:02.365 INFO:teuthology.orchestra.run.smithi175.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:07:02.365 INFO:teuthology.orchestra.run.smithi175.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:07:02.366 INFO:teuthology.orchestra.run.smithi175.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:07:02.366 INFO:teuthology.orchestra.run.smithi150.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:07:02.367 INFO:teuthology.orchestra.run.smithi175.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:07:02.367 INFO:teuthology.orchestra.run.smithi150.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:07:02.368 INFO:teuthology.orchestra.run.smithi175.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:07:02.368 INFO:teuthology.orchestra.run.smithi175.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:07:02.368 INFO:teuthology.orchestra.run.smithi175.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:07:02.370 INFO:teuthology.orchestra.run.smithi150.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:07:02.371 INFO:teuthology.orchestra.run.smithi150.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:07:02.377 INFO:teuthology.orchestra.run.smithi150.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:07:02.381 INFO:teuthology.orchestra.run.smithi175.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:07:02.381 INFO:teuthology.orchestra.run.smithi175.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:07:02.392 INFO:teuthology.orchestra.run.smithi150.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:07:02.393 INFO:teuthology.orchestra.run.smithi150.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:07:02.394 INFO:teuthology.orchestra.run.smithi150.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:07:02.394 INFO:teuthology.orchestra.run.smithi150.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:07:02.398 INFO:teuthology.orchestra.run.smithi175.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:07:02.401 INFO:teuthology.orchestra.run.smithi175.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:07:02.401 INFO:teuthology.orchestra.run.smithi175.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:07:02.402 INFO:teuthology.orchestra.run.smithi175.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:07:02.402 INFO:teuthology.orchestra.run.smithi175.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:07:02.403 INFO:teuthology.orchestra.run.smithi175.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:07:02.403 INFO:teuthology.orchestra.run.smithi175.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:07:02.404 INFO:teuthology.orchestra.run.smithi175.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:07:02.404 INFO:teuthology.orchestra.run.smithi150.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:07:02.415 INFO:teuthology.orchestra.run.smithi175.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:07:02.416 INFO:teuthology.orchestra.run.smithi175.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:07:02.422 INFO:teuthology.orchestra.run.smithi150.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:07:02.424 INFO:teuthology.orchestra.run.smithi150.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:07:02.424 INFO:teuthology.orchestra.run.smithi150.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:07:02.432 INFO:teuthology.orchestra.run.smithi175.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:07:02.438 INFO:teuthology.orchestra.run.smithi175.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:07:02.438 INFO:teuthology.orchestra.run.smithi175.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:07:02.438 INFO:teuthology.orchestra.run.smithi175.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:07:02.439 INFO:teuthology.orchestra.run.smithi175.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:07:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:07:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:07:02.440 INFO:teuthology.orchestra.run.smithi175.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:07:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:07:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:07:02.442 INFO:teuthology.orchestra.run.smithi150.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:07:02.442 INFO:teuthology.orchestra.run.smithi175.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:07:02.442 INFO:teuthology.orchestra.run.smithi150.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:07:02.443 INFO:teuthology.orchestra.run.smithi175.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:07:02.445 INFO:teuthology.orchestra.run.smithi150.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:07:02.446 INFO:teuthology.orchestra.run.smithi150.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:07:02.449 INFO:teuthology.orchestra.run.smithi150.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:07:02.449 INFO:teuthology.orchestra.run.smithi175.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:07:02.450 INFO:teuthology.orchestra.run.smithi175.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:07:02.457 INFO:teuthology.orchestra.run.smithi175.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:07:02.462 INFO:teuthology.orchestra.run.smithi150.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:07:02.464 INFO:teuthology.orchestra.run.smithi150.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:07:02.464 INFO:teuthology.orchestra.run.smithi150.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:07:02.466 INFO:teuthology.orchestra.run.smithi175.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:07:02.467 INFO:teuthology.orchestra.run.smithi175.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:07:02.467 INFO:teuthology.orchestra.run.smithi150.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:07:02.467 INFO:teuthology.orchestra.run.smithi175.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:07:02.470 INFO:teuthology.orchestra.run.smithi150.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:07:02.470 INFO:teuthology.orchestra.run.smithi175.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:07:02.470 INFO:teuthology.orchestra.run.smithi150.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:07:02.471 INFO:teuthology.orchestra.run.smithi150.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:07:02.479 INFO:teuthology.orchestra.run.smithi150.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:07:02.480 INFO:teuthology.orchestra.run.smithi150.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:07:02.483 INFO:teuthology.orchestra.run.smithi150.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:07:02.496 INFO:teuthology.orchestra.run.smithi150.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:07:02.497 INFO:teuthology.orchestra.run.smithi150.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:07:02.497 INFO:teuthology.orchestra.run.smithi150.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:07:02.499 INFO:teuthology.orchestra.run.smithi150.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:07:02.500 INFO:teuthology.orchestra.run.smithi150.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:07:02.506 INFO:teuthology.orchestra.run.smithi150.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:07:02.506 INFO:teuthology.orchestra.run.smithi150.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:07:02.513 INFO:teuthology.orchestra.run.smithi150.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:07:02.515 INFO:teuthology.orchestra.run.smithi150.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:07:02.516 INFO:teuthology.orchestra.run.smithi150.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:07:02.523 INFO:teuthology.orchestra.run.smithi175.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:07:02.524 INFO:teuthology.orchestra.run.smithi175.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:07:02.525 INFO:teuthology.orchestra.run.smithi175.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:07:02.525 INFO:teuthology.orchestra.run.smithi175.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:07:02.526 INFO:teuthology.orchestra.run.smithi175.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:07:02.543 INFO:teuthology.orchestra.run.smithi175.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:07:02.545 INFO:teuthology.orchestra.run.smithi175.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:07:02.545 INFO:teuthology.orchestra.run.smithi175.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:07:02.560 INFO:teuthology.orchestra.run.smithi175.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:07:02.560 INFO:teuthology.orchestra.run.smithi175.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:07:02.560 INFO:teuthology.orchestra.run.smithi175.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:07:02.561 INFO:teuthology.orchestra.run.smithi175.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:07:02.561 INFO:teuthology.orchestra.run.smithi175.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:07:02.562 INFO:teuthology.orchestra.run.smithi175.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:07:02.567 INFO:teuthology.orchestra.run.smithi175.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:07:02.576 INFO:teuthology.orchestra.run.smithi175.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:07:02.577 INFO:teuthology.orchestra.run.smithi175.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:07:02.580 INFO:teuthology.orchestra.run.smithi175.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:07:02.582 INFO:teuthology.orchestra.run.smithi150.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:07:02.585 INFO:teuthology.orchestra.run.smithi150.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:07:02.593 INFO:teuthology.orchestra.run.smithi175.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:07:02.594 INFO:teuthology.orchestra.run.smithi150.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:07:02.595 INFO:teuthology.orchestra.run.smithi175.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:07:02.595 INFO:teuthology.orchestra.run.smithi175.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:07:02.598 INFO:teuthology.orchestra.run.smithi175.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:07:02.601 INFO:teuthology.orchestra.run.smithi175.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:07:02.601 INFO:teuthology.orchestra.run.smithi175.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:07:02.602 INFO:teuthology.orchestra.run.smithi175.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:07:02.604 INFO:teuthology.orchestra.run.smithi175.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:07:02.612 INFO:teuthology.orchestra.run.smithi175.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:07:02.612 INFO:teuthology.orchestra.run.smithi175.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:07:02.613 INFO:teuthology.orchestra.run.smithi175.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:07:02.614 INFO:teuthology.orchestra.run.smithi175.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:07:02.615 INFO:teuthology.orchestra.run.smithi175.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:07:02.617 INFO:teuthology.orchestra.run.smithi175.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:07:02.619 INFO:teuthology.orchestra.run.smithi175.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:07:02.626 INFO:teuthology.orchestra.run.smithi175.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:07:02.628 INFO:teuthology.orchestra.run.smithi175.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:07:02.629 INFO:teuthology.orchestra.run.smithi175.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:07:02.629 INFO:teuthology.orchestra.run.smithi175.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:07:02.631 INFO:teuthology.orchestra.run.smithi175.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:07:02.631 INFO:teuthology.orchestra.run.smithi175.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:07:02.637 INFO:teuthology.orchestra.run.smithi175.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:07:02.637 INFO:teuthology.orchestra.run.smithi175.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:07:02.645 INFO:teuthology.orchestra.run.smithi175.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:07:02.647 INFO:teuthology.orchestra.run.smithi175.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:07:02.648 INFO:teuthology.orchestra.run.smithi175.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:07:02.900 INFO:teuthology.orchestra.run.smithi175.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:07:02.902 INFO:teuthology.orchestra.run.smithi175.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:07:02.912 INFO:teuthology.orchestra.run.smithi175.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:07:03.481 INFO:teuthology.orchestra.run.smithi150.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:07:03.556 INFO:teuthology.orchestra.run.smithi150.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:07:03.559 INFO:teuthology.orchestra.run.smithi150.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:07:03.561 INFO:teuthology.orchestra.run.smithi150.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:07:03.581 INFO:teuthology.orchestra.run.smithi150.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:07:03.734 INFO:teuthology.orchestra.run.smithi150.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:07:04.020 INFO:teuthology.orchestra.run.smithi175.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:07:04.193 INFO:teuthology.orchestra.run.smithi175.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:07:04.198 INFO:teuthology.orchestra.run.smithi175.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:07:04.213 INFO:teuthology.orchestra.run.smithi175.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:07:04.275 INFO:teuthology.orchestra.run.smithi175.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:07:04.429 INFO:teuthology.orchestra.run.smithi150.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:07:04.430 INFO:teuthology.orchestra.run.smithi150.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:07:04.447 INFO:teuthology.orchestra.run.smithi150.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:07:04.475 INFO:teuthology.orchestra.run.smithi175.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:07:04.490 INFO:teuthology.orchestra.run.smithi150.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:07:04.501 INFO:teuthology.orchestra.run.smithi150.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:07:04.502 INFO:teuthology.orchestra.run.smithi150.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:07:04.536 INFO:teuthology.orchestra.run.smithi150.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:07:04.765 INFO:teuthology.orchestra.run.smithi150.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:07:04.765 INFO:teuthology.orchestra.run.smithi150.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:07:05.515 INFO:teuthology.orchestra.run.smithi175.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:07:05.515 INFO:teuthology.orchestra.run.smithi175.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:07:05.533 INFO:teuthology.orchestra.run.smithi175.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:07:05.574 INFO:teuthology.orchestra.run.smithi175.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:07:05.614 INFO:teuthology.orchestra.run.smithi175.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:07:05.617 INFO:teuthology.orchestra.run.smithi175.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:07:05.715 INFO:teuthology.orchestra.run.smithi175.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:07:06.057 INFO:teuthology.orchestra.run.smithi175.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:07:06.057 INFO:teuthology.orchestra.run.smithi175.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:07:06.407 INFO:teuthology.orchestra.run.smithi150.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:07:06.409 INFO:teuthology.orchestra.run.smithi150.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:07:06.411 INFO:teuthology.orchestra.run.smithi150.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:07:06.720 INFO:teuthology.orchestra.run.smithi150.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:07:07.287 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 181 MB in 5s (38.0 MB/s) 2024-09-21T23:07:07.426 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:07:07.448 INFO:teuthology.orchestra.run.smithi175.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:07:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:07:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:07:07.525 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:07:07.531 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:07:07.616 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:07:07.641 INFO:teuthology.orchestra.run.smithi175.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:07:07.895 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:07:07.921 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:07:07.953 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:07:08.216 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 181 MB in 6s (32.6 MB/s) 2024-09-21T23:07:08.272 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:07:08.298 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:07:08.339 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:07:08.376 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:07:08.476 INFO:teuthology.orchestra.run.smithi175.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:07:08.481 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:07:08.567 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:07:08.792 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:07:08.819 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:07:08.867 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:07:08.920 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:07:08.946 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:07:08.986 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:07:09.237 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:07:09.263 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:07:09.298 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:07:09.303 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:07:09.324 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:07:09.364 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:07:09.648 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:07:09.675 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:07:09.715 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:07:09.801 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:07:09.827 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:07:09.876 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:07:10.060 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:07:10.086 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:07:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:07:10.312 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:07:10.338 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:07:10.379 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:07:10.412 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librados2. 2024-09-21T23:07:10.427 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:10.462 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:10.699 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:07:10.725 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:07:10.765 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:07:10.907 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:07:10.933 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:10.973 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:11.101 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:07:11.128 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:07:11.168 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:07:11.393 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:07:11.419 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:11.459 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:11.487 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librados2. 2024-09-21T23:07:11.513 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:11.704 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:07:11.719 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:11.761 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:11.982 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:07:11.997 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:12.031 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:12.081 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:07:12.107 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:12.148 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:12.400 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:07:12.427 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:12.459 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:07:12.467 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:12.485 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:12.525 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:12.761 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:07:12.788 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:12.803 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:07:12.828 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:12.829 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:12.869 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:13.147 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:07:13.174 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:07:13.189 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:07:13.205 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:07:13.215 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:13.255 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:13.491 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:07:13.506 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:13.508 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:07:13.535 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:07:13.566 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:13.575 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:07:13.836 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:07:13.852 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:07:13.862 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:13.878 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:13.902 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:13.918 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:14.213 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:07:14.239 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:07:14.264 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:07:14.279 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:07:14.290 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:07:14.330 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:07:14.600 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:07:14.615 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:07:14.678 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:07:14.877 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:07:14.903 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:07:14.949 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:07:15.136 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:07:15.162 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:07:15.202 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:07:15.202 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:07:15.217 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:15.252 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:15.505 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:07:15.532 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:07:15.563 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:07:15.572 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:07:15.589 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:07:15.629 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:07:15.883 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:07:15.890 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:07:15.899 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:07:15.905 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:07:15.933 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:07:15.948 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:07:16.153 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package zip. 2024-09-21T23:07:16.179 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:07:16.219 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:07:16.251 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:07:16.277 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:07:16.318 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:07:16.531 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package unzip. 2024-09-21T23:07:16.558 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:07:16.614 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:07:16.646 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:07:16.672 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:07:16.712 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:07:17.001 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:07:17.028 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:07:17.032 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:07:17.058 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:07:17.068 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:07:17.099 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:07:17.351 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package zip. 2024-09-21T23:07:17.366 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:07:17.409 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:07:17.434 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:07:17.449 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:17.484 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:17.721 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package unzip. 2024-09-21T23:07:17.747 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:07:17.787 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:07:18.019 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:07:18.035 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:18.078 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:18.183 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:07:18.209 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:07:18.249 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:07:18.406 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:07:18.432 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:07:18.472 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:07:18.632 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:07:18.658 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:18.698 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:18.767 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:07:18.794 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:18.833 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:19.220 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:07:19.226 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:07:19.247 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:19.252 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:19.286 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:19.292 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:19.637 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:07:19.663 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:07:19.703 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:07:20.031 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:07:20.058 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:20.098 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:20.434 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:07:20.462 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:20.492 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:20.825 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:07:20.852 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:20.901 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:21.471 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:07:21.497 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:07:21.537 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:07:21.807 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:07:21.822 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:07:21.833 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:07:21.848 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:21.865 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:07:21.898 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:22.152 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:07:22.178 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:07:22.218 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:07:22.485 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:07:22.511 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:07:22.521 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:07:22.547 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:07:22.551 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:07:22.587 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:07:22.854 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:07:22.873 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:07:22.880 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:07:22.899 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:07:22.920 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:07:22.948 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:07:23.210 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:07:23.236 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:07:23.240 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:07:23.267 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:07:23.268 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:07:23.307 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:07:23.546 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:07:23.572 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:07:23.585 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:07:23.600 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:07:23.613 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:07:23.635 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:07:23.904 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:07:23.915 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:07:23.930 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:07:23.941 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:07:23.970 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:07:23.982 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:07:24.265 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:07:24.276 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:07:24.292 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:07:24.303 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:07:24.343 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:07:24.348 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:07:24.646 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:07:24.651 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:07:24.673 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:07:24.678 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:07:24.713 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:07:24.717 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:07:25.024 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:07:25.051 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:07:25.108 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:07:25.187 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:07:25.213 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:07:25.245 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:07:25.427 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:07:25.454 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:07:25.494 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:07:25.523 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:07:25.549 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:07:25.589 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:07:25.771 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:07:25.798 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:07:25.838 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:07:25.901 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:07:25.928 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:07:25.959 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:07:26.149 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:07:26.165 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:07:26.208 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:07:26.254 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:07:26.280 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:07:26.320 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:07:26.494 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:07:26.520 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:07:26.561 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:07:26.623 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:07:26.650 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:07:26.689 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:07:26.846 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:07:26.873 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:07:26.913 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:07:26.992 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:07:27.019 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:07:27.059 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:07:27.258 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:07:27.284 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:07:27.327 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:07:27.387 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:07:27.414 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:07:27.453 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:07:27.602 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:07:27.617 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:07:27.652 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:07:27.731 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:07:27.758 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:07:27.798 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:07:27.896 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:07:27.912 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:07:27.946 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:07:28.150 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:07:28.177 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:07:28.217 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:07:28.258 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:07:28.284 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:07:28.333 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:07:28.553 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:07:28.577 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:07:28.580 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:07:28.604 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:07:28.623 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:07:28.635 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:07:28.915 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:07:28.939 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:07:28.941 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:07:28.965 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:07:28.989 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:07:29.005 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:07:29.275 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:07:29.291 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:07:29.316 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:07:29.325 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:07:29.342 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:07:29.383 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:07:29.628 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:07:29.655 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:07:29.695 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:07:29.771 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:07:29.797 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:07:29.837 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:07:29.980 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:07:30.007 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:07:30.047 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:07:30.165 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:07:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:07:30.232 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:07:30.334 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:07:30.360 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:07:30.400 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:07:30.518 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:07:30.545 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:07:30.585 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:07:30.711 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:07:30.738 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:07:30.778 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:07:30.871 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:07:30.898 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:07:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:07:31.124 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:07:31.151 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:07:31.190 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:07:31.249 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:07:31.275 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:07:31.315 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:07:31.501 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:07:31.517 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:07:31.551 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:07:31.560 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:07:31.575 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:31.619 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:31.854 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:07:31.881 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:07:31.921 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:07:31.990 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:07:32.016 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:32.056 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:32.224 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:07:32.251 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:07:32.290 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:07:32.345 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:07:32.371 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:07:32.409 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:07:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:07:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:07:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:07:32.712 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:07:32.738 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:07:32.795 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:07:32.937 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:07:32.953 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:32.995 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:33.131 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:07:33.158 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:33.198 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:33.384 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:07:33.411 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:33.450 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:33.560 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:07:33.586 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:33.643 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:33.720 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:07:33.747 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:07:33.778 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:07:34.072 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:07:34.099 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:07:34.138 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:07:34.213 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:07:34.239 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:34.279 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:35.324 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph. 2024-09-21T23:07:35.340 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:35.382 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:35.457 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:07:35.484 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:35.534 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:35.904 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:07:35.930 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:35.970 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:36.574 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:07:36.586 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:07:36.600 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:36.602 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:36.636 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:36.640 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:37.005 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:07:37.033 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:37.072 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:37.475 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:07:37.502 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:37.541 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:37.676 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph. 2024-09-21T23:07:37.703 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:37.743 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:37.853 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:07:37.880 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:07:37.919 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:07:38.062 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:07:38.088 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:38.128 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:38.222 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:07:38.249 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:07:38.288 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:07:38.465 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:07:38.492 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:38.531 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:38.943 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:07:38.969 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:39.009 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:39.268 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:07:39.296 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:07:39.320 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:07:39.334 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:07:39.347 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:07:39.395 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:07:39.646 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:07:39.673 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:07:39.697 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:07:39.712 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:07:39.713 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:07:39.764 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:07:40.049 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:07:40.076 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:40.115 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:40.460 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:07:40.487 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:07:40.527 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:07:40.819 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:07:40.838 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:07:40.846 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:07:40.865 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:07:40.885 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:07:40.904 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:07:41.188 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:07:41.215 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:07:41.240 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:07:41.254 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:07:41.268 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:41.307 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:41.599 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:07:41.626 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:41.666 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:42.002 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:07:42.029 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:07:42.069 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:07:42.363 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:07:42.378 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:07:42.413 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:07:42.522 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:07:42.539 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:07:42.581 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:07:42.749 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:07:42.776 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:42.816 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:43.043 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:07:43.070 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:07:43.109 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:07:43.471 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:07:43.498 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:07:43.537 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:07:43.848 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:07:43.875 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:07:43.915 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:07:44.037 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:07:44.064 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:07:44.104 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:07:44.523 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:07:44.539 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:44.616 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:07:44.642 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:07:44.659 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:44.682 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:07:45.052 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:07:45.079 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:07:45.118 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:07:45.404 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:07:45.431 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:07:45.470 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:07:45.596 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:07:45.622 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:07:45.662 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:07:45.948 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:07:45.975 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:07:46.014 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:07:46.301 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:07:46.317 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:07:46.351 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:07:46.406 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:07:46.433 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:46.558 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:46.646 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:07:46.677 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:07:46.871 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:07:47.174 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:07:47.201 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:07:47.241 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:07:47.479 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:07:47.505 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:07:47.535 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:07:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:07:47.563 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:07:47.593 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:07:47.823 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:07:47.850 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:07:47.889 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:07:47.905 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:07:47.932 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:07:47.971 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:07:48.226 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:07:48.253 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:07:48.292 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:07:48.300 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:07:48.327 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:07:48.366 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:07:48.587 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:07:48.614 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:07:48.654 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:07:48.966 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:07:48.993 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:07:49.030 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:07:49.041 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:07:49.046 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:49.088 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:49.319 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:07:49.335 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:07:49.369 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:07:49.416 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:07:49.444 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:07:49.482 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:07:49.672 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:07:49.699 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:07:49.738 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:07:49.819 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:07:49.846 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:07:49.885 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:07:50.059 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:07:50.086 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:07:50.121 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package jq. 2024-09-21T23:07:50.125 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:07:50.145 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:07:50.179 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:07:50.415 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package socat. 2024-09-21T23:07:50.443 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:07:50.482 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:07:50.789 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:07:50.794 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:07:50.817 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:50.821 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:07:50.855 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:50.860 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:07:51.192 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:07:51.219 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:07:51.231 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:07:51.258 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:07:51.258 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:51.297 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:51.587 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:07:51.614 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:07:51.653 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:07:51.889 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package jq. 2024-09-21T23:07:51.916 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:07:51.955 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:07:52.166 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package socat. 2024-09-21T23:07:52.182 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:07:52.233 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:07:52.528 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:07:52.555 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:07:52.594 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:07:52.949 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:07:52.964 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:53.015 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:53.552 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:07:53.580 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:53.618 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:53.947 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:07:53.963 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:54.005 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:54.342 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:07:54.370 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:07:54.408 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:07:54.763 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:07:54.791 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:07:54.830 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:07:55.183 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:07:55.211 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:07:55.249 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:07:55.311 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:07:55.339 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:07:55.378 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:55.671 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:07:55.690 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:07:55.699 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:07:55.717 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:07:55.738 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:07:55.756 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:07:56.149 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:07:56.160 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:07:56.177 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:07:56.187 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:07:56.216 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:07:56.234 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:07:56.572 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:07:56.600 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:07:56.635 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:07:56.638 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:07:56.663 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:07:56.702 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:07:57.013 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:07:57.018 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:07:57.041 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:07:57.045 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:07:57.079 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:07:57.084 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:07:57.383 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:07:57.411 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:07:57.449 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:07:57.506 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:07:57.534 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:07:57.572 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:07:57.711 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:07:57.738 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:07:57.777 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:07:57.842 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:07:57.870 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:07:57.909 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:07:58.113 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:07:58.141 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:07:58.196 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:07:58.312 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:07:58.339 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:07:58.378 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:07:58.483 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:07:58.510 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:07:58.541 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:07:58.681 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:07:58.709 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:07:58.747 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:07:58.979 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:07:59.007 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:07:59.045 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:07:59.076 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:07:59.104 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:07:59.142 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:07:59.356 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:07:59.384 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:07:59.422 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:07:59.454 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:07:59.482 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:07:59.520 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:07:59.725 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:07:59.753 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:07:59.792 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:07:59.873 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:07:59.901 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:07:59.940 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:08:00.145 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:08:00.173 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:08:00.212 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:08:00.293 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:08:00.321 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:08:00.360 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:08:00.506 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:08:00.522 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:08:00.564 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:08:00.831 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:08:00.858 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:08:00.897 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:08:01.141 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:08:01.170 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:08:01.216 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:08:01.293 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:08:01.322 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:08:01.359 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:01.761 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:08:01.789 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:08:01.827 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:08:02.147 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:08:02.175 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:08:02.214 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:08:02.446 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:08:02.463 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:08:02.504 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:02.508 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:08:02.524 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:08:02.566 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:08:02.800 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:08:03.051 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:08:03.168 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:08:03.296 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:08:03.324 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:08:03.362 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:03.433 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:08:03.776 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:08:04.165 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:08:04.173 INFO:teuthology.orchestra.run.smithi150.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:08:04.193 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:08:04.231 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:04.236 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:04.511 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:08:04.745 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user cephadm....done 2024-09-21T23:08:04.770 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:08:04.893 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:08:04.896 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:08:05.160 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:08:05.160 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:08:05.420 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:08:05.487 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:08:05.547 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:08:05.806 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:08:05.914 INFO:teuthology.orchestra.run.smithi175.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:08:05.997 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:06.065 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:08:06.183 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:08:06.468 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:08:06.514 INFO:teuthology.orchestra.run.smithi175.stdout:Adding system user cephadm....done 2024-09-21T23:08:06.670 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:08:06.828 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:08:06.946 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:08:07.088 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:08:07.206 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:08:07.356 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:08:07.357 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:08:07.608 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:08:07.649 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:08:07.851 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:08:07.883 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:08:07.977 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:08:08.126 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:08:08.279 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:08:08.368 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:08.630 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:08:08.636 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:08:08.882 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:08:08.971 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:08:09.097 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:08:09.150 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:08:09.223 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:08:09.468 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:08:09.474 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:08:09.719 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:08:09.869 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:08:09.900 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:08:09.960 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:08:09.969 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:08:10.086 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:08:10.221 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:10.454 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:08:10.489 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:08:10.588 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:08:10.615 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:08:10.741 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:08:10.839 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:08:10.867 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:08:11.099 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:08:11.143 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:08:11.400 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:08:11.501 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:08:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:08:11.724 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:08:11.780 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:08:11.905 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:08:12.178 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:08:12.265 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:08:12.399 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:08:12.438 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:08:12.563 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:08:12.650 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:08:12.689 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:08:12.910 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:08:12.982 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:08:13.220 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:08:13.346 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:08:13.375 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:08:13.609 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:08:13.998 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:08:14.002 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:08:14.257 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:08:14.321 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:08:14.358 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:08:14.484 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:08:14.664 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:08:14.777 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:08:14.907 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:08:15.041 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:15.137 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:08:15.334 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:08:15.379 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:08:15.643 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:08:15.773 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:08:16.066 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:08:16.401 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:08:16.643 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:08:16.769 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:16.840 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:08:17.100 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:08:17.234 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:08:17.294 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:08:17.494 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:08:17.620 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:08:17.663 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:08:18.029 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:08:18.322 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:08:18.565 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:08:18.786 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:08:18.824 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:08:18.950 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:08:19.038 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:08:19.172 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:08:19.243 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:08:19.431 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:08:19.503 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:08:19.541 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:08:19.667 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:08:19.771 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:08:19.952 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:08:20.013 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:08:20.203 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:08:20.248 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:08:20.446 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:08:20.515 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:08:20.572 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:08:20.625 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:08:20.873 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:08:20.892 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:08:21.019 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:08:21.108 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:08:21.320 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:08:21.359 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:08:21.604 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:08:21.618 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:08:21.855 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:08:21.886 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:08:21.956 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:08:22.082 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:08:22.154 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:08:22.171 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:08:22.280 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:08:22.401 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:08:22.518 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:08:22.556 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:08:22.690 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:08:22.920 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:08:22.967 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:08:23.179 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:08:23.251 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:08:23.297 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:08:23.511 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:08:23.548 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:08:23.637 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:08:23.674 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:08:23.762 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:08:23.852 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:08:23.975 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:08:24.114 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:08:24.224 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:08:24.276 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:08:24.560 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:08:24.592 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:08:24.686 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:08:24.843 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:08:24.963 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:08:24.969 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:08:25.214 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:08:25.255 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:08:25.371 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:08:25.550 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:08:25.689 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:08:25.927 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:08:25.982 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:08:26.220 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:08:26.258 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:08:26.394 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:08:26.581 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:08:26.660 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:08:26.699 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:26.825 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:26.928 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:08:26.984 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:08:27.256 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:08:27.320 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:27.455 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:27.572 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:08:27.650 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:08:27.841 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:27.926 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:08:27.966 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:28.093 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:28.262 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:08:28.306 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:08:28.306 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:08:28.388 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:28.514 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:28.649 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:08:28.836 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:28.962 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:28.968 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:29.090 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:29.094 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:29.216 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:08:29.220 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:08:29.513 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:29.639 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:29.774 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:29.985 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:08:29.985 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:08:30.436 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:30.466 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:30.562 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:30.593 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:30.688 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:30.719 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:30.797 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:30.845 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:08:30.924 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:31.500 INFO:teuthology.orchestra.run.smithi150.stdout:Adding group ceph....done 2024-09-21T23:08:31.810 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user ceph....done 2024-09-21T23:08:31.987 INFO:teuthology.orchestra.run.smithi150.stdout:Setting system user ceph properties....done 2024-09-21T23:08:31.999 INFO:teuthology.orchestra.run.smithi150.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:08:32.000 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:32.118 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:32.143 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:08:32.244 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:32.370 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:32.502 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:08:32.529 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:32.967 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:33.088 INFO:teuthology.orchestra.run.smithi175.stdout:Adding group ceph....done 2024-09-21T23:08:33.134 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:33.432 INFO:teuthology.orchestra.run.smithi175.stdout:Adding system user ceph....done 2024-09-21T23:08:33.600 INFO:teuthology.orchestra.run.smithi175.stdout:Setting system user ceph properties....done 2024-09-21T23:08:33.619 INFO:teuthology.orchestra.run.smithi175.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:08:33.632 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:08:33.632 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:08:33.754 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:08:34.132 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:08:34.146 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:34.451 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:08:34.655 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:34.789 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:34.914 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:35.140 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:08:35.140 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:08:35.289 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:08:35.289 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:08:35.641 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:35.741 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:35.860 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:08:35.860 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:08:36.202 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:08:36.342 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:36.619 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:08:36.619 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:08:36.718 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:36.957 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:08:36.958 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:08:37.127 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:37.253 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:37.404 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:37.420 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:37.598 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:08:37.598 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:08:37.653 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:08:37.653 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:08:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:38.147 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:38.231 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:38.358 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:38.415 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:08:38.415 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:08:38.687 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:38.931 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:39.031 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:08:39.049 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:39.166 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:08:39.175 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:39.318 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:08:39.386 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:08:39.386 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:08:39.893 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:40.027 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:40.128 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:40.271 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:08:40.402 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:08:40.624 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:08:40.732 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:08:40.885 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:08:41.060 INFO:teuthology.orchestra.run.smithi150.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:08:41.066 DEBUG:teuthology.parallel:result is None 2024-09-21T23:08:41.651 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:08:42.297 INFO:teuthology.orchestra.run.smithi175.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:08:42.301 DEBUG:teuthology.parallel:result is None 2024-09-21T23:08:42.302 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:08:42.452 DEBUG:teuthology.orchestra.run.smithi150:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:08:42.471 INFO:teuthology.orchestra.run.smithi150.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:08:42.472 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:08:42.472 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:08:42.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:08:42.620 DEBUG:teuthology.orchestra.run.smithi175:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:08:42.639 INFO:teuthology.orchestra.run.smithi175.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:08:42.639 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:08:42.639 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:08:42.641 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T23:08:42.641 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:42.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:08:42.656 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:08:42.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:08:42.694 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T23:08:42.694 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:42.694 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:08:42.713 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:08:42.768 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:08:42.768 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:08:42.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:08:42.833 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T23:08:42.834 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:42.834 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:08:42.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:08:42.904 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:08:42.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:08:42.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:08:42.973 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T23:08:42.974 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:42.974 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:08:42.987 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:08:43.039 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:08:43.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:08:43.053 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:08:43.109 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T23:08:43.202 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T23:08:43.202 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2024-09-21T23:08:43.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2024-09-21T23:08:43.217 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T23:08:43.217 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:08:43.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:08:43.273 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T23:08:43.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:08:43.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:08:43.340 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T23:08:43.340 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T23:08:43.340 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914506', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1478, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '24/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEc63V0g2edqYC8QUmQeCrgtINQVX4mxm/M5jzALBN1ZatwWDIQTkVcaqybmtAKY7ScUgqIfdVqFt8ZccNgdmoA=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNvwl4cIDU8XsIRA5ZHXmUbzJCfcrQIsMg/t/Id+xBqgwaHnqOHyOYrNc7mqWYLJN/O+uZ24F2ignLoqiiHV1i0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T23:08:43.341 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:08:43.369 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:08:43.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:08:43.417 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:08:43.434 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:43.434 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T23:08:43.476 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T23:08:43.476 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 23:03:23.104773316 +0000 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 23:03:22.876775352 +0000 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 23:03:22.876775352 +0000 2024-09-21T23:08:43.525 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-21T23:08:43.525 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T23:08:43.579 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T23:08:43.579 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T23:08:43.579 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000452681 s, 1.1 MB/s 2024-09-21T23:08:43.580 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T23:08:43.630 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 23:03:23.612768779 +0000 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 23:03:23.344771172 +0000 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 23:03:23.344771172 +0000 2024-09-21T23:08:43.677 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-21T23:08:43.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T23:08:43.731 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T23:08:43.731 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T23:08:43.731 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000385117 s, 1.3 MB/s 2024-09-21T23:08:43.732 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T23:08:43.781 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-09-21T23:08:43.828 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T23:08:43.829 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:43.829 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-21T23:08:43.829 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:43.829 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 23:03:24.092764491 +0000 2024-09-21T23:08:43.829 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 23:03:23.844766706 +0000 2024-09-21T23:08:43.829 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 23:03:23.844766706 +0000 2024-09-21T23:08:43.829 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-21T23:08:43.829 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T23:08:43.883 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T23:08:43.883 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T23:08:43.883 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000337323 s, 1.5 MB/s 2024-09-21T23:08:43.884 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T23:08:43.934 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-09-21T23:08:43.980 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T23:08:43.980 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:43.981 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-09-21T23:08:43.981 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:43.981 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-21 23:03:24.556760347 +0000 2024-09-21T23:08:43.981 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-21 23:03:24.336762312 +0000 2024-09-21T23:08:43.981 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-21 23:03:24.336762312 +0000 2024-09-21T23:08:43.981 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-21T23:08:43.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T23:08:44.035 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-21T23:08:44.035 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-21T23:08:44.035 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000330715 s, 1.5 MB/s 2024-09-21T23:08:44.036 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T23:08:44.085 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T23:08:44.085 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:08:44.085 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T23:08:44.091 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T23:08:44.091 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-09-21T23:08:44.138 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T23:08:44.138 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:44.138 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-09-21T23:08:44.138 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:44.138 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 23:03:21.677551106 +0000 2024-09-21T23:08:44.138 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 23:03:21.429553321 +0000 2024-09-21T23:08:44.139 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 23:03:21.429553321 +0000 2024-09-21T23:08:44.139 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-09-21T23:08:44.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T23:08:44.192 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T23:08:44.193 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T23:08:44.193 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000445196 s, 1.2 MB/s 2024-09-21T23:08:44.194 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T23:08:44.244 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-09-21T23:08:44.290 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T23:08:44.290 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:44.290 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-09-21T23:08:44.290 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:44.290 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 23:03:22.161546782 +0000 2024-09-21T23:08:44.291 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 23:03:21.901549104 +0000 2024-09-21T23:08:44.291 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 23:03:21.901549104 +0000 2024-09-21T23:08:44.291 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-09-21T23:08:44.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T23:08:44.344 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T23:08:44.344 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T23:08:44.344 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.0004459 s, 1.1 MB/s 2024-09-21T23:08:44.346 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T23:08:44.395 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-09-21T23:08:44.442 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T23:08:44.442 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:44.443 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-09-21T23:08:44.443 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:44.443 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 23:03:22.637542531 +0000 2024-09-21T23:08:44.443 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 23:03:22.389544746 +0000 2024-09-21T23:08:44.443 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 23:03:22.389544746 +0000 2024-09-21T23:08:44.443 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-09-21T23:08:44.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T23:08:44.495 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T23:08:44.496 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T23:08:44.496 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000324617 s, 1.6 MB/s 2024-09-21T23:08:44.497 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T23:08:44.543 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 23:03:23.157537885 +0000 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 23:03:22.905540137 +0000 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 23:03:22.905540137 +0000 2024-09-21T23:08:44.590 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-09-21T23:08:44.590 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T23:08:44.644 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T23:08:44.644 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T23:08:44.644 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000402718 s, 1.3 MB/s 2024-09-21T23:08:44.645 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T23:08:44.696 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T23:08:44.696 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T23:08:44.696 INFO:tasks.ceph:Generating config... 2024-09-21T23:08:44.698 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-09-21T23:08:44.698 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-09-21T23:08:44.698 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-09-21T23:08:44.698 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T23:08:44.699 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T23:08:44.699 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T23:08:44.699 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T23:08:44.699 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T23:08:44.699 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T23:08:44.699 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T23:08:44.700 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T23:08:44.700 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T23:08:44.700 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T23:08:44.700 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-21T23:08:44.700 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T23:08:44.700 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T23:08:44.700 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T23:08:44.701 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T23:08:44.701 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T23:08:44.701 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T23:08:44.701 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T23:08:44.701 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T23:08:44.701 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T23:08:44.702 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T23:08:44.702 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T23:08:44.702 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T23:08:44.702 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T23:08:44.702 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T23:08:44.702 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-21T23:08:44.702 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T23:08:44.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T23:08:44.737 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T23:08:44.747 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T23:08:44.822 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:08:44.876 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.150'), ('mon.c', '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]'), ('mon.b', '172.21.15.175')] 2024-09-21T23:08:44.876 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790],172.21.15.175', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T23:08:44.877 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:44.877 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T23:08:44.925 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.150 --addv c '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]' --add b 172.21.15.175 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: generated fsid 9ce7cfc1-a7d6-426f-bf44-a308fb883b8f 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:setting min_mon_release = quincy 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 0 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:fsid 9ce7cfc1-a7d6-426f-bf44-a308fb883b8f 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:last_changed 2024-09-21T23:08:45.011916+0000 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-09-21T23:08:45.011916+0000 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:min_mon_release 17 (quincy) 2024-09-21T23:08:45.011 INFO:teuthology.orchestra.run.smithi150.stdout:election_strategy: 1 2024-09-21T23:08:45.012 INFO:teuthology.orchestra.run.smithi150.stdout:0: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.a 2024-09-21T23:08:45.012 INFO:teuthology.orchestra.run.smithi150.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T23:08:45.012 INFO:teuthology.orchestra.run.smithi150.stdout:2: [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] mon.c 2024-09-21T23:08:45.012 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T23:08:45.013 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T23:08:45.061 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9ce7cfc1-a7d6-426f-bf44-a308fb883b8f... 2024-09-21T23:08:45.062 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:08:45.105 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.134 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-09-21T23:08:45.135 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790],172.21.15.175" 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: mon client directed command retry = 5 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on pool no app = False 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject socket failures = 5000 2024-09-21T23:08:45.136 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 9ce7cfc1-a7d6-426f-bf44-a308fb883b8f 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.137 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-09-21T23:08:45.138 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-09-21T23:08:45.139 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2024-09-21T23:08:45.140 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2024-09-21T23:08:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-21T23:08:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-09-21T23:08:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2024-09-21T23:08:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:08:45.141 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd max markdown count = 1000 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: osd shutdown pgref assert = True 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.142 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.143 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-09-21T23:08:45.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:08:45.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-09-21T23:08:45.146 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout: rbd qos iops burst = 100 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout: rbd qos iops limit = 50 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout: rbd qos schedule tick min = 100 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-09-21T23:08:45.147 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-09-21T23:08:45.149 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-09-21T23:08:45.149 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.150 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.151 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:08:45.152 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790],172.21.15.175" 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: mon client directed command retry = 5 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on pool no app = False 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject socket failures = 5000 2024-09-21T23:08:45.153 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = 9ce7cfc1-a7d6-426f-bf44-a308fb883b8f 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-09-21T23:08:45.154 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug trim objects = true 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-09-21T23:08:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-09-21T23:08:45.156 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: osd max markdown count = 1000 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: osd shutdown pgref assert = True 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:08:45.157 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.158 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-09-21T23:08:45.159 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout: rbd qos iops burst = 100 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout: rbd qos iops limit = 50 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout: rbd qos schedule tick min = 100 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-09-21T23:08:45.160 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-09-21T23:08:45.160 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T23:08:45.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T23:08:45.245 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T23:08:45.246 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:45.246 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T23:08:45.251 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:45.251 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T23:08:45.296 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:08:45.296 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:45.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:08:45.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:08:45.359 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:08:45.360 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:45.408 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:08:45.409 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:08:45.409 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:08:45.409 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:08:45.433 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:08:45.433 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:45.482 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T23:08:45.482 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T23:08:45.482 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T23:08:45.525 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T23:08:45.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T23:08:45.571 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T23:08:45.581 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T23:08:45.581 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T23:08:45.582 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T23:08:45.615 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T23:08:45.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T23:08:45.657 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T23:08:45.667 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T23:08:45.667 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T23:08:45.667 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T23:08:45.679 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T23:08:45.679 INFO:tasks.ceph:role: osd.0 2024-09-21T23:08:45.679 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:08:45.679 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:08:45.740 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:08:46.220 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T23:08:46.222 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T23:08:46.222 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T23:08:46.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T23:08:46.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:08:46.353 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:08:46.354 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T23:08:46.408 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T23:08:46.408 INFO:tasks.ceph:role: osd.1 2024-09-21T23:08:46.408 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:08:46.408 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T23:08:46.472 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:08:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:08:46.473 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:08:46.912 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T23:08:46.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T23:08:46.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T23:08:46.977 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T23:08:47.028 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:08:47.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:08:47.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T23:08:47.084 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T23:08:47.084 INFO:tasks.ceph:role: osd.2 2024-09-21T23:08:47.084 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:08:47.085 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T23:08:47.148 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:08:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:08:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:08:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:08:47.148 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:08:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:08:47.149 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:08:47.149 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:08:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:08:47.149 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:08:47.609 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T23:08:47.611 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T23:08:47.611 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T23:08:47.699 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T23:08:47.712 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:08:47.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:08:47.713 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T23:08:47.768 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T23:08:47.769 INFO:tasks.ceph:role: osd.3 2024-09-21T23:08:47.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:08:47.769 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T23:08:47.833 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:08:47.833 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:08:47.834 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:08:48.309 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-21T23:08:48.311 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-21T23:08:48.311 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T23:08:48.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T23:08:48.489 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:08:48.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:08:48.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:48.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:48.602+0000 7f4a08e427c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T23:08:48.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:48.606+0000 7f4a08e427c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T23:08:48.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:48.606+0000 7f4a08e427c0 -1 bdev(0x55c9df605800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T23:08:48.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:48.606+0000 7f4a08e427c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T23:08:52.020 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T23:08:52.034 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:52.114+0000 7fa9169ed7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T23:08:52.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:52.114+0000 7fa9169ed7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T23:08:52.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:52.114+0000 7fa9169ed7c0 -1 bdev(0x55a44dadd800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T23:08:52.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:52.114+0000 7fa9169ed7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T23:08:55.245 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T23:08:55.301 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:55.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:55.378+0000 7f52912ec7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T23:08:55.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:55.382+0000 7f52912ec7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T23:08:55.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:55.382+0000 7f52912ec7c0 -1 bdev(0x555faf10f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T23:08:55.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:55.382+0000 7f52912ec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T23:08:58.532 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T23:08:58.589 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:08:58.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:58.666+0000 7f21199207c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T23:08:58.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:58.670+0000 7f21199207c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T23:08:58.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:58.670+0000 7f21199207c0 -1 bdev(0x5613b4aeb800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T23:08:58.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:08:58.670+0000 7f21199207c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T23:09:01.835 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T23:09:01.891 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T23:09:01.905 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T23:09:01.906 INFO:tasks.ceph:role: osd.4 2024-09-21T23:09:01.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:09:01.906 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:09:01.976 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:09:01.977 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:09:01.977 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:09:02.456 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T23:09:02.459 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T23:09:02.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T23:09:02.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T23:09:02.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:09:02.617 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:09:02.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T23:09:02.673 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T23:09:02.674 INFO:tasks.ceph:role: osd.5 2024-09-21T23:09:02.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:09:02.674 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T23:09:02.736 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:09:02.737 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:09:03.194 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T23:09:03.197 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T23:09:03.197 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T23:09:03.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T23:09:03.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:09:03.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:09:03.300 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T23:09:03.352 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T23:09:03.352 INFO:tasks.ceph:role: osd.6 2024-09-21T23:09:03.352 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:09:03.352 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T23:09:03.414 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:09:03.414 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:09:03.414 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:09:03.414 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:09:03.414 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:09:03.414 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:09:03.414 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:09:03.415 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:09:03.415 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:09:03.415 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:09:03.878 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T23:09:03.882 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T23:09:03.882 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T23:09:04.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T23:09:04.028 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:09:04.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:09:04.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T23:09:04.086 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T23:09:04.086 INFO:tasks.ceph:role: osd.7 2024-09-21T23:09:04.086 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:09:04.086 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:09:04.150 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:09:04.151 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:09:04.151 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:09:04.645 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T23:09:04.647 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T23:09:04.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T23:09:04.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T23:09:04.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:09:04.817 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:09:04.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:09:04.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:04.934+0000 7f44efed77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T23:09:04.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:04.934+0000 7f44efed77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T23:09:04.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:04.934+0000 7f44efed77c0 -1 bdev(0x557379919800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T23:09:04.936 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:04.934+0000 7f44efed77c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T23:09:08.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T23:09:08.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:09:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:08.210+0000 7fbb7d37f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T23:09:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:08.210+0000 7fbb7d37f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T23:09:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:08.210+0000 7fbb7d37f7c0 -1 bdev(0x5561aeb75800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T23:09:08.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:08.210+0000 7fbb7d37f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T23:09:11.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T23:09:11.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:09:11.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:11.510+0000 7ff2779a37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T23:09:11.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:11.510+0000 7ff2779a37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T23:09:11.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:11.510+0000 7ff2779a37c0 -1 bdev(0x5638195f7800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T23:09:11.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:11.510+0000 7ff2779a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T23:09:14.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T23:09:14.735 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:09:14.816 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:14.814+0000 7f64e67377c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T23:09:14.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:14.814+0000 7f64e67377c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T23:09:14.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:14.814+0000 7f64e67377c0 -1 bdev(0x55bd2713f800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T23:09:14.817 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T23:09:14.814+0000 7f64e67377c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T23:09:18.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T23:09:18.235 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T23:09:18.235 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:18.235 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T23:09:18.250 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:18.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T23:09:18.304 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:18.304 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T23:09:18.359 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:18.359 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T23:09:18.415 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:18.415 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T23:09:18.471 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:18.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T23:09:18.485 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:18.485 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T23:09:18.541 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:18.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T23:09:18.596 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:18.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T23:09:18.648 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:18.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T23:09:18.701 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:18.701 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T23:09:18.706 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:18.707 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T23:09:18.750 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T23:09:18.750 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T23:09:18.752 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T23:09:18.795 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2024-09-21T23:09:18.795 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD9Ue9mXaRlHxAAX40n4+DjIhsOBMyDdPKDaQ== 2024-09-21T23:09:18.795 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAAUu9mlCk0JBAAtCqNbjWJtz9dHJLZXTIomg== 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAEUu9mpJMVBxAAqZoZ3TPHdwwH28ndMHO49g== 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAHUu9mCkryFhAArzjAgZgrUH0ldObBLcABjA== 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAKUu9m2hMZKBAA/Vlz0niNs6Xs7MV8O/0klQ== 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD9Ue9mqkc8IhAAHl4HoOTnZ43+nb5csI92HQ== 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAQUu9mhZ3dNxAAdDOtN+cAD5Ygvpp3zvepIw== 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAUUu9mFa3VDBAAvP0eMyZXa7Pv2q5omjcE8A== 2024-09-21T23:09:18.796 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2024-09-21T23:09:18.797 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAXUu9mBWW9HhAAeux0VQQTlDQrP4mo27xz/Q== 2024-09-21T23:09:18.797 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2024-09-21T23:09:18.797 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAaUu9mXYrbMBAAQ/zTDdvLNOa6NDansEFnUg== 2024-09-21T23:09:18.797 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-09-21T23:09:18.797 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD9Ue9m2oG2JBAAtchCj37clSiNQco0TA2pkA== 2024-09-21T23:09:18.797 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-09-21T23:09:18.797 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD9Ue9meHlfJxAA24vJzEI7ty3YoTrwvOP/0w== 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD9Ue9mXaRlHxAAX40n4+DjIhsOBMyDdPKDaQ== 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAAUu9mlCk0JBAAtCqNbjWJtz9dHJLZXTIomg== 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAEUu9mpJMVBxAAqZoZ3TPHdwwH28ndMHO49g== 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAHUu9mCkryFhAArzjAgZgrUH0ldObBLcABjA== 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAKUu9m2hMZKBAA/Vlz0niNs6Xs7MV8O/0klQ== 2024-09-21T23:09:18.804 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD9Ue9mqkc8IhAAHl4HoOTnZ43+nb5csI92HQ== 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAQUu9mhZ3dNxAAdDOtN+cAD5Ygvpp3zvepIw== 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAUUu9mFa3VDBAAvP0eMyZXa7Pv2q5omjcE8A== 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAXUu9mBWW9HhAAeux0VQQTlDQrP4mo27xz/Q== 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQAaUu9mXYrbMBAAQ/zTDdvLNOa6NDansEFnUg== 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD9Ue9m2oG2JBAAtchCj37clSiNQco0TA2pkA== 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-09-21T23:09:18.805 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQD9Ue9meHlfJxAA24vJzEI7ty3YoTrwvOP/0w== 2024-09-21T23:09:18.806 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:09:18.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:09:18.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:18.896 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:18.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:18.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.068 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.148 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.229 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:09:19.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:09:19.317 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.405 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.577 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:09:19.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:09:19.661 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:09:19.708 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:09:19.745 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:09:19.791 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T23:09:19.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T23:09:19.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:09:19.927 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T23:09:19.938 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T23:09:19.989 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:09:20.080 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T23:09:20.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T23:09:20.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:09:20.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T23:09:20.216 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:09:20.218 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:09:20.261 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T23:09:20.261 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T23:09:20.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T23:09:20.265 INFO:tasks.ceph.mon.a:Started 2024-09-21T23:09:20.265 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T23:09:20.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T23:09:20.267 INFO:tasks.ceph.mon.c:Started 2024-09-21T23:09:20.268 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T23:09:20.268 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T23:09:20.307 INFO:tasks.ceph.mon.b:Started 2024-09-21T23:09:20.307 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T23:09:20.307 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T23:09:20.307 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T23:09:20.310 INFO:tasks.ceph.mgr.y:Started 2024-09-21T23:09:20.310 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T23:09:20.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T23:09:20.312 INFO:tasks.ceph.mgr.x:Started 2024-09-21T23:09:20.312 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T23:09:20.313 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T23:09:25.969 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:25.966+0000 7f37b808c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T23:09:26.049 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:26.046+0000 7f38fbb84180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T23:09:26.508 INFO:teuthology.orchestra.run.smithi150.stderr:adjusted tunables profile to default 2024-09-21T23:09:26.517 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:26.514+0000 7f37b808c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T23:09:26.527 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T23:09:26.527 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T23:09:26.528 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:26.528 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T23:09:26.538 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:26.539 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T23:09:26.552 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:26.550+0000 7f38fbb84180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T23:09:26.561 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:26.561 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T23:09:26.613 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-21T23:09:26.613 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T23:09:26.664 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:26.664 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T23:09:26.677 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:26.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T23:09:26.733 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:26.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T23:09:26.746 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:26.742+0000 7f38fbb84180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T23:09:26.748 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:26.746+0000 7f37b808c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T23:09:26.758 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T23:09:26.759 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T23:09:26.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d 0 2024-09-21T23:09:27.135 INFO:teuthology.orchestra.run.smithi175.stdout:0 2024-09-21T23:09:27.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 5fb69564-3b15-470e-b45f-f70594751344 1 2024-09-21T23:09:27.297 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.294+0000 7f38fbb84180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T23:09:27.351 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.346+0000 7f38fbb84180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T23:09:27.361 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T23:09:27.377 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new cc3e3e88-a324-4e3d-8b40-0d8c93da1400 2 2024-09-21T23:09:27.404 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.402+0000 7f38fbb84180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T23:09:27.412 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.410+0000 7f37b808c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T23:09:27.470 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.466+0000 7f37b808c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T23:09:27.480 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.478+0000 7f38fbb84180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T23:09:27.525 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.522+0000 7f37b808c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T23:09:27.533 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.530+0000 7f38fbb84180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T23:09:27.588 INFO:teuthology.orchestra.run.smithi175.stdout:2 2024-09-21T23:09:27.606 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 933fe042-8aed-4795-918c-572cf81999a7 3 2024-09-21T23:09:27.608 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.606+0000 7f37b808c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T23:09:27.611 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.606+0000 7f38fbb84180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T23:09:27.660 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.658+0000 7f37b808c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T23:09:27.664 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.662+0000 7f38fbb84180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T23:09:27.743 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.742+0000 7f37b808c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T23:09:27.797 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.794+0000 7f37b808c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T23:09:27.814 INFO:teuthology.orchestra.run.smithi175.stdout:3 2024-09-21T23:09:27.816 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.814+0000 7f38fbb84180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T23:09:27.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new af826961-2dad-4702-a904-991e2f5f57c8 4 2024-09-21T23:09:27.877 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:27.874+0000 7f38fbb84180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T23:09:27.955 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:27.954+0000 7f37b808c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T23:09:28.018 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:28.018+0000 7f37b808c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T23:09:28.030 INFO:teuthology.orchestra.run.smithi175.stdout:4 2024-09-21T23:09:28.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new c4f1908b-0885-408e-873e-4d2aa9b9c506 5 2024-09-21T23:09:28.243 INFO:teuthology.orchestra.run.smithi175.stdout:5 2024-09-21T23:09:28.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 08cef493-ce38-4b52-b5f2-23e7694953c0 6 2024-09-21T23:09:28.488 INFO:teuthology.orchestra.run.smithi175.stdout:6 2024-09-21T23:09:28.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 26d2142e-419c-4da3-9c28-ac442b1b66fb 7 2024-09-21T23:09:28.576 INFO:tasks.ceph.mgr.y.smithi150.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T23:09:28.576 INFO:tasks.ceph.mgr.y.smithi150.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T23:09:28.577 INFO:tasks.ceph.mgr.y.smithi150.stderr: from numpy import show_config as show_numpy_config 2024-09-21T23:09:28.590 INFO:tasks.ceph.mgr.x.smithi175.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T23:09:28.590 INFO:tasks.ceph.mgr.x.smithi175.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T23:09:28.590 INFO:tasks.ceph.mgr.x.smithi175.stderr: from numpy import show_config as show_numpy_config 2024-09-21T23:09:28.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:28.634+0000 7f37b808c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T23:09:28.650 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:28.646+0000 7f38fbb84180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T23:09:28.695 INFO:teuthology.orchestra.run.smithi175.stdout:7 2024-09-21T23:09:28.712 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T23:09:28.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T23:09:28.714 INFO:tasks.ceph.osd.0:Started 2024-09-21T23:09:28.714 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T23:09:28.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T23:09:28.717 INFO:tasks.ceph.osd.1:Started 2024-09-21T23:09:28.717 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T23:09:28.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T23:09:28.719 INFO:tasks.ceph.osd.2:Started 2024-09-21T23:09:28.719 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T23:09:28.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T23:09:28.722 INFO:tasks.ceph.osd.3:Started 2024-09-21T23:09:28.722 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T23:09:28.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T23:09:28.724 INFO:tasks.ceph.osd.4:Started 2024-09-21T23:09:28.724 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T23:09:28.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T23:09:28.726 INFO:tasks.ceph.osd.5:Started 2024-09-21T23:09:28.726 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T23:09:28.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T23:09:28.728 INFO:tasks.ceph.osd.6:Started 2024-09-21T23:09:28.728 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T23:09:28.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T23:09:28.730 INFO:tasks.ceph.osd.7:Started 2024-09-21T23:09:28.731 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:09:28.803 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:28.802+0000 7f37b808c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:09:28.872 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:28.870+0000 7f38fbb84180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:09:28.923 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:28.923 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":10,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:09:28.692303+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:09:28.692303+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:09:28.939 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T23:09:28.939 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T23:09:28.962 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:28.962+0000 7f37b808c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T23:09:29.025 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:29.022+0000 7f37b808c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T23:09:29.034 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:29.030+0000 7f38fbb84180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T23:09:29.103 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:29.098+0000 7f38fbb84180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T23:09:29.160 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:29.158+0000 7f37b808c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:09:29.250 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:29.246+0000 7f38fbb84180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:09:29.744 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:09:29.767 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:29.766+0000 7f37b808c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T23:09:29.843 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:29.842+0000 7f37b808c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T23:09:29.874 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:29.870+0000 7f38fbb84180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T23:09:29.918 INFO:teuthology.misc.health.smithi150.stdout: 2024-09-21T23:09:29.918 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":10,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:09:28.692303+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:09:28.692303+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:09:29.933 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T23:09:29.951 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:29.950+0000 7f38fbb84180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T23:09:30.026 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:30.026+0000 7f37b808c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T23:09:30.100 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:30.098+0000 7f37b808c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T23:09:30.134 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:30.130+0000 7f38fbb84180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T23:09:30.138 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:30.138+0000 7f48b3e5f7c0 -1 Falling back to public interface 2024-09-21T23:09:30.207 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:30.206+0000 7f38fbb84180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T23:09:30.394 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:30.394+0000 7f37b808c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T23:09:30.410 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:30.410+0000 7f2d739f17c0 -1 Falling back to public interface 2024-09-21T23:09:30.416 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:09:30.414+0000 7ff8534e87c0 -1 Falling back to public interface 2024-09-21T23:09:30.420 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:30.418+0000 7f6b865f47c0 -1 Falling back to public interface 2024-09-21T23:09:30.456 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:30.454+0000 7f37b808c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T23:09:30.502 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:30.498+0000 7f38fbb84180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T23:09:30.569 INFO:tasks.ceph.mgr.y.smithi150.stderr:2024-09-21T23:09:30.566+0000 7f38fbb84180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T23:09:30.655 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:30.654+0000 7f06c97057c0 -1 Falling back to public interface 2024-09-21T23:09:30.661 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:30.658+0000 7f0d199e97c0 -1 Falling back to public interface 2024-09-21T23:09:30.883 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:30.882+0000 7f0e536ed7c0 -1 Falling back to public interface 2024-09-21T23:09:30.909 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:30.906+0000 7f3ed285e7c0 -1 Falling back to public interface 2024-09-21T23:09:31.499 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:31.498+0000 7f375ebc0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T23:09:31.499 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T23:09:31.498+0000 7f375ebc0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T23:09:34.661 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:34.658+0000 7f2d739f17c0 -1 osd.7 0 log_to_monitors true 2024-09-21T23:09:34.881 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:34.878+0000 7f06c97057c0 -1 osd.6 0 log_to_monitors true 2024-09-21T23:09:34.908 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:34.906+0000 7f3ed285e7c0 -1 osd.1 0 log_to_monitors true 2024-09-21T23:09:34.938 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:34.938+0000 7f48b3e5f7c0 -1 osd.4 0 log_to_monitors true 2024-09-21T23:09:34.954 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:09:34.950+0000 7ff8534e87c0 -1 osd.2 0 log_to_monitors true 2024-09-21T23:09:34.976 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:34.974+0000 7f0d199e97c0 -1 osd.0 0 log_to_monitors true 2024-09-21T23:09:35.005 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:35.002+0000 7f6b865f47c0 -1 osd.3 0 log_to_monitors true 2024-09-21T23:09:35.349 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:35.350+0000 7f0e536ed7c0 -1 osd.5 0 log_to_monitors true 2024-09-21T23:09:36.740 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:09:37.055 INFO:teuthology.misc.health.smithi150.stdout: 2024-09-21T23:09:37.055 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":12,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:09:36.503514+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:09:28.692303+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:09:37.072 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T23:09:38.525 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:38.526+0000 7f0e4ee73640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T23:09:38.574 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:38.574+0000 7f0e474ba640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:38.625 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:38.626+0000 7f06c4e8b640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T23:09:38.645 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:38.646+0000 7f48b061a640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T23:09:38.657 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:38.658+0000 7f06bd4d2640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:38.673 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:38.674+0000 7f48a7c2c640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:38.682 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:38.682+0000 7f2d6f177640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T23:09:38.714 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:38.714+0000 7f2d677be640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:38.727 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:38.726+0000 7f3ecdfe4640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T23:09:38.741 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:38.738+0000 7f6b82daf640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T23:09:38.748 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:38.746+0000 7f0d1516f640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T23:09:38.752 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:38.750+0000 7f3ec662b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:38.769 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:09:38.766+0000 7ff84ec6e640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T23:09:38.772 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:38.770+0000 7f6b7a3c1640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:38.784 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:38.782+0000 7f0d0d7b6640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:38.795 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:09:38.794+0000 7ff8472b5640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:09:43.878 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:09:44.294 INFO:teuthology.misc.health.smithi150.stdout: 2024-09-21T23:09:44.295 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":16,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:09:42.658501+0000","last_up_change":"2024-09-21T23:09:39.531143+0000","last_in_change":"2024-09-21T23:09:28.692303+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:09:40.569134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6817","nonce":198356244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6819","nonce":198356244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6823","nonce":198356244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6821","nonce":198356244}]},"public_addr":"172.21.15.150:6817/198356244","cluster_addr":"172.21.15.150:6819/198356244","heartbeat_back_addr":"172.21.15.150:6823/198356244","heartbeat_front_addr":"172.21.15.150:6821/198356244","state":["exists","up"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6825","nonce":725319294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6827","nonce":725319294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6831","nonce":725319294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6829","nonce":725319294}]},"public_addr":"172.21.15.150:6825/725319294","cluster_addr":"172.21.15.150:6827/725319294","heartbeat_back_addr":"172.21.15.150:6831/725319294","heartbeat_front_addr":"172.21.15.150:6829/725319294","state":["exists","up"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6801","nonce":1956153436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6803","nonce":1956153436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6807","nonce":1956153436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6805","nonce":1956153436}]},"public_addr":"172.21.15.150:6801/1956153436","cluster_addr":"172.21.15.150:6803/1956153436","heartbeat_back_addr":"172.21.15.150:6807/1956153436","heartbeat_front_addr":"172.21.15.150:6805/1956153436","state":["exists","up"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6809","nonce":3049870482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6811","nonce":3049870482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6815","nonce":3049870482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6813","nonce":3049870482}]},"public_addr":"172.21.15.150:6809/3049870482","cluster_addr":"172.21.15.150:6811/3049870482","heartbeat_back_addr":"172.21.15.150:6815/3049870482","heartbeat_front_addr":"172.21.15.150:6813/3049870482","state":["exists","up"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6801","nonce":4112998952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6803","nonce":4112998952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6807","nonce":4112998952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6805","nonce":4112998952}]},"public_addr":"172.21.15.175:6801/4112998952","cluster_addr":"172.21.15.175:6803/4112998952","heartbeat_back_addr":"172.21.15.175:6807/4112998952","heartbeat_front_addr":"172.21.15.175:6805/4112998952","state":["exists","up"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6827","nonce":120048378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6829","nonce":120048378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6833","nonce":120048378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6831","nonce":120048378}]},"public_addr":"172.21.15.175:6827/120048378","cluster_addr":"172.21.15.175:6829/120048378","heartbeat_back_addr":"172.21.15.175:6833/120048378","heartbeat_front_addr":"172.21.15.175:6831/120048378","state":["exists","up"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6819","nonce":787648514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6821","nonce":787648514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6825","nonce":787648514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6823","nonce":787648514}]},"public_addr":"172.21.15.175:6819/787648514","cluster_addr":"172.21.15.175:6821/787648514","heartbeat_back_addr":"172.21.15.175:6825/787648514","heartbeat_front_addr":"172.21.15.175:6823/787648514","state":["exists","up"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6809","nonce":1424462696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6811","nonce":1424462696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6815","nonce":1424462696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6813","nonce":1424462696}]},"public_addr":"172.21.15.175:6809/1424462696","cluster_addr":"172.21.15.175:6811/1424462696","heartbeat_back_addr":"172.21.15.175:6815/1424462696","heartbeat_front_addr":"172.21.15.175:6813/1424462696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.004700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.958497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.988302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.993971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.945648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.366302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.852613+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.713697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:09:44.312 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T23:09:44.312 INFO:tasks.ceph:Creating RBD pool 2024-09-21T23:09:44.312 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T23:09:44.683 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'rbd' created 2024-09-21T23:09:44.707 DEBUG:teuthology.orchestra.run.smithi150:> rbd --cluster ceph pool init rbd 2024-09-21T23:09:47.745 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T23:09:47.746 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T23:09:47.746 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T23:09:47.746 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T23:09:47.746 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T23:09:48.099 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:48.115 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":3908983421},{"type":"v1","addr":"172.21.15.175:6817","nonce":3908983421}]},"active_addr":"172.21.15.175:6817/3908983421","active_change":"2024-09-21T23:09:30.476550+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2754829102}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":1235398995}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3116756325}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2680170318}]}]} 2024-09-21T23:09:48.118 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T23:09:48.118 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T23:09:48.118 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:09:48.400 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:48.400 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":20,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:09:47.708037+0000","last_up_change":"2024-09-21T23:09:39.531143+0000","last_in_change":"2024-09-21T23:09:28.692303+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:09:40.569134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:09:44.626156+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6817","nonce":198356244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6819","nonce":198356244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6823","nonce":198356244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6821","nonce":198356244}]},"public_addr":"172.21.15.150:6817/198356244","cluster_addr":"172.21.15.150:6819/198356244","heartbeat_back_addr":"172.21.15.150:6823/198356244","heartbeat_front_addr":"172.21.15.150:6821/198356244","state":["exists","up"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6825","nonce":725319294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6827","nonce":725319294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6831","nonce":725319294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6829","nonce":725319294}]},"public_addr":"172.21.15.150:6825/725319294","cluster_addr":"172.21.15.150:6827/725319294","heartbeat_back_addr":"172.21.15.150:6831/725319294","heartbeat_front_addr":"172.21.15.150:6829/725319294","state":["exists","up"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6801","nonce":1956153436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6803","nonce":1956153436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6807","nonce":1956153436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6805","nonce":1956153436}]},"public_addr":"172.21.15.150:6801/1956153436","cluster_addr":"172.21.15.150:6803/1956153436","heartbeat_back_addr":"172.21.15.150:6807/1956153436","heartbeat_front_addr":"172.21.15.150:6805/1956153436","state":["exists","up"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6809","nonce":3049870482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6811","nonce":3049870482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6815","nonce":3049870482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6813","nonce":3049870482}]},"public_addr":"172.21.15.150:6809/3049870482","cluster_addr":"172.21.15.150:6811/3049870482","heartbeat_back_addr":"172.21.15.150:6815/3049870482","heartbeat_front_addr":"172.21.15.150:6813/3049870482","state":["exists","up"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6801","nonce":4112998952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6803","nonce":4112998952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6807","nonce":4112998952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6805","nonce":4112998952}]},"public_addr":"172.21.15.175:6801/4112998952","cluster_addr":"172.21.15.175:6803/4112998952","heartbeat_back_addr":"172.21.15.175:6807/4112998952","heartbeat_front_addr":"172.21.15.175:6805/4112998952","state":["exists","up"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6827","nonce":120048378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6829","nonce":120048378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6833","nonce":120048378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6831","nonce":120048378}]},"public_addr":"172.21.15.175:6827/120048378","cluster_addr":"172.21.15.175:6829/120048378","heartbeat_back_addr":"172.21.15.175:6833/120048378","heartbeat_front_addr":"172.21.15.175:6831/120048378","state":["exists","up"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6819","nonce":787648514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6821","nonce":787648514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6825","nonce":787648514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6823","nonce":787648514}]},"public_addr":"172.21.15.175:6819/787648514","cluster_addr":"172.21.15.175:6821/787648514","heartbeat_back_addr":"172.21.15.175:6825/787648514","heartbeat_front_addr":"172.21.15.175:6823/787648514","state":["exists","up"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6809","nonce":1424462696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6811","nonce":1424462696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6815","nonce":1424462696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6813","nonce":1424462696}]},"public_addr":"172.21.15.175:6809/1424462696","cluster_addr":"172.21.15.175:6811/1424462696","heartbeat_back_addr":"172.21.15.175:6815/1424462696","heartbeat_front_addr":"172.21.15.175:6813/1424462696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.004700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.958497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.988302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.993971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.945648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.366302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.852613+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.713697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:09:48.419 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T23:09:48.419 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:09:48.706 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:48.706 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":20,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:09:47.708037+0000","last_up_change":"2024-09-21T23:09:39.531143+0000","last_in_change":"2024-09-21T23:09:28.692303+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:09:40.569134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:09:44.626156+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6817","nonce":198356244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6819","nonce":198356244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6823","nonce":198356244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6821","nonce":198356244}]},"public_addr":"172.21.15.150:6817/198356244","cluster_addr":"172.21.15.150:6819/198356244","heartbeat_back_addr":"172.21.15.150:6823/198356244","heartbeat_front_addr":"172.21.15.150:6821/198356244","state":["exists","up"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6825","nonce":725319294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6827","nonce":725319294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6831","nonce":725319294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6829","nonce":725319294}]},"public_addr":"172.21.15.150:6825/725319294","cluster_addr":"172.21.15.150:6827/725319294","heartbeat_back_addr":"172.21.15.150:6831/725319294","heartbeat_front_addr":"172.21.15.150:6829/725319294","state":["exists","up"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6801","nonce":1956153436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6803","nonce":1956153436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6807","nonce":1956153436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6805","nonce":1956153436}]},"public_addr":"172.21.15.150:6801/1956153436","cluster_addr":"172.21.15.150:6803/1956153436","heartbeat_back_addr":"172.21.15.150:6807/1956153436","heartbeat_front_addr":"172.21.15.150:6805/1956153436","state":["exists","up"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6809","nonce":3049870482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6811","nonce":3049870482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6815","nonce":3049870482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6813","nonce":3049870482}]},"public_addr":"172.21.15.150:6809/3049870482","cluster_addr":"172.21.15.150:6811/3049870482","heartbeat_back_addr":"172.21.15.150:6815/3049870482","heartbeat_front_addr":"172.21.15.150:6813/3049870482","state":["exists","up"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6801","nonce":4112998952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6803","nonce":4112998952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6807","nonce":4112998952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":4112998952},{"type":"v1","addr":"172.21.15.175:6805","nonce":4112998952}]},"public_addr":"172.21.15.175:6801/4112998952","cluster_addr":"172.21.15.175:6803/4112998952","heartbeat_back_addr":"172.21.15.175:6807/4112998952","heartbeat_front_addr":"172.21.15.175:6805/4112998952","state":["exists","up"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6827","nonce":120048378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6829","nonce":120048378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6833","nonce":120048378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6831","nonce":120048378}]},"public_addr":"172.21.15.175:6827/120048378","cluster_addr":"172.21.15.175:6829/120048378","heartbeat_back_addr":"172.21.15.175:6833/120048378","heartbeat_front_addr":"172.21.15.175:6831/120048378","state":["exists","up"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6819","nonce":787648514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6821","nonce":787648514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6825","nonce":787648514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6823","nonce":787648514}]},"public_addr":"172.21.15.175:6819/787648514","cluster_addr":"172.21.15.175:6821/787648514","heartbeat_back_addr":"172.21.15.175:6825/787648514","heartbeat_front_addr":"172.21.15.175:6823/787648514","state":["exists","up"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6809","nonce":1424462696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6811","nonce":1424462696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6815","nonce":1424462696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6813","nonce":1424462696}]},"public_addr":"172.21.15.175:6809/1424462696","cluster_addr":"172.21.15.175:6811/1424462696","heartbeat_back_addr":"172.21.15.175:6815/1424462696","heartbeat_front_addr":"172.21.15.175:6813/1424462696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.004700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.958497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.988302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.993971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.945648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.366302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.852613+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.713697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:09:48.724 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T23:09:48.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T23:09:48.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T23:09:48.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T23:09:48.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T23:09:48.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T23:09:48.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T23:09:48.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T23:09:48.987 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:48.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:09:48.992 INFO:teuthology.orchestra.run.smithi150.stdout:55834574854 2024-09-21T23:09:48.993 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T23:09:48.999 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:48.999 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:09:49.003 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:49.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:09:49.005 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:49.006 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:49.006 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:09:49.007 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:09:49.008 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:49.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:09:49.012 INFO:teuthology.orchestra.run.smithi150.stdout:55834574854 2024-09-21T23:09:49.012 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T23:09:49.495 INFO:teuthology.orchestra.run.smithi150.stdout:55834574851 2024-09-21T23:09:49.513 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:49.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2024-09-21T23:09:49.523 INFO:teuthology.orchestra.run.smithi150.stdout:55834574851 2024-09-21T23:09:49.530 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:49.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-09-21T23:09:49.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-09-21T23:09:49.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-09-21T23:09:49.561 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:49.561 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:49.563 INFO:teuthology.orchestra.run.smithi150.stdout:55834574851 2024-09-21T23:09:49.583 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:49.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-09-21T23:09:49.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-09-21T23:09:49.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-09-21T23:09:49.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-09-21T23:09:50.521 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:09:50.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:09:50.555 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:09:50.560 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T23:09:50.585 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:09:50.586 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:09:50.591 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T23:09:50.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:09:51.026 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:51.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-09-21T23:09:51.048 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.067 INFO:teuthology.orchestra.run.smithi150.stdout:55834574854 2024-09-21T23:09:51.085 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:51.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-09-21T23:09:51.093 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.100 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:55834574854 2024-09-21T23:09:51.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-09-21T23:09:51.110 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-09-21T23:09:51.123 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-21T23:09:51.129 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.132 INFO:teuthology.orchestra.run.smithi150.stdout:55834574852 2024-09-21T23:09:51.149 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:51.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-09-21T23:09:51.149 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.161 INFO:teuthology.orchestra.run.smithi150.stdout:55834574853 2024-09-21T23:09:51.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-09-21T23:09:51.172 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-09-21T23:09:51.180 DEBUG:teuthology.parallel:result is None 2024-09-21T23:09:51.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:09:51.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:09:51.495 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:51.495 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:09:51.512 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-21T23:09:50.488295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219008,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2834432,"data_stored":1731942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.785860"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067730+0000","last_change":"2024-09-21T23:09:48.068366+0000","last_active":"2024-09-21T23:09:48.067730+0000","last_peered":"2024-09-21T23:09:48.067730+0000","last_clean":"2024-09-21T23:09:48.067730+0000","last_became_active":"2024-09-21T23:09:45.718621+0000","last_became_peered":"2024-09-21T23:09:45.718621+0000","last_unstale":"2024-09-21T23:09:48.067730+0000","last_undegraded":"2024-09-21T23:09:48.067730+0000","last_fullsized":"2024-09-21T23:09:48.067730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:44:48.782244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725647+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:09:47.725647+0000","last_peered":"2024-09-21T23:09:47.725647+0000","last_clean":"2024-09-21T23:09:47.725647+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:09:47.725647+0000","last_undegraded":"2024-09-21T23:09:47.725647+0000","last_fullsized":"2024-09-21T23:09:47.725647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068127+0000","last_change":"2024-09-21T23:09:48.068555+0000","last_active":"2024-09-21T23:09:48.068127+0000","last_peered":"2024-09-21T23:09:48.068127+0000","last_clean":"2024-09-21T23:09:48.068127+0000","last_became_active":"2024-09-21T23:09:45.869065+0000","last_became_peered":"2024-09-21T23:09:45.869065+0000","last_unstale":"2024-09-21T23:09:48.068127+0000","last_undegraded":"2024-09-21T23:09:48.068127+0000","last_fullsized":"2024-09-21T23:09:48.068127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:48.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088130999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725811+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:09:47.725811+0000","last_peered":"2024-09-21T23:09:47.725811+0000","last_clean":"2024-09-21T23:09:47.725811+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:09:47.725811+0000","last_undegraded":"2024-09-21T23:09:47.725811+0000","last_fullsized":"2024-09-21T23:09:47.725811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.724225+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:09:47.724225+0000","last_peered":"2024-09-21T23:09:47.724225+0000","last_clean":"2024-09-21T23:09:47.724225+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:09:47.724225+0000","last_undegraded":"2024-09-21T23:09:47.724225+0000","last_fullsized":"2024-09-21T23:09:47.724225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.716035+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:09:47.716035+0000","last_peered":"2024-09-21T23:09:47.716035+0000","last_clean":"2024-09-21T23:09:47.716035+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:09:47.716035+0000","last_undegraded":"2024-09-21T23:09:47.716035+0000","last_fullsized":"2024-09-21T23:09:47.716035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068015+0000","last_change":"2024-09-21T23:09:48.068542+0000","last_active":"2024-09-21T23:09:48.068015+0000","last_peered":"2024-09-21T23:09:48.068015+0000","last_clean":"2024-09-21T23:09:48.068015+0000","last_became_active":"2024-09-21T23:09:45.714516+0000","last_became_peered":"2024-09-21T23:09:45.714516+0000","last_unstale":"2024-09-21T23:09:48.068015+0000","last_undegraded":"2024-09-21T23:09:48.068015+0000","last_fullsized":"2024-09-21T23:09:48.068015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:51.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000999572,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.722583+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:09:47.722583+0000","last_peered":"2024-09-21T23:09:47.722583+0000","last_clean":"2024-09-21T23:09:47.722583+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:09:47.722583+0000","last_undegraded":"2024-09-21T23:09:47.722583+0000","last_fullsized":"2024-09-21T23:09:47.722583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067436+0000","last_change":"2024-09-21T23:09:41.675036+0000","last_active":"2024-09-21T23:09:48.067436+0000","last_peered":"2024-09-21T23:09:48.067436+0000","last_clean":"2024-09-21T23:09:48.067436+0000","last_became_active":"2024-09-21T23:09:41.674018+0000","last_became_peered":"2024-09-21T23:09:41.674018+0000","last_unstale":"2024-09-21T23:09:48.067436+0000","last_undegraded":"2024-09-21T23:09:48.067436+0000","last_fullsized":"2024-09-21T23:09:48.067436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:10.788395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:09:51.513 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:09:51.782 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:51.782 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:09:51.800 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-21T23:09:50.488295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219008,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2834432,"data_stored":1731942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.785860"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067730+0000","last_change":"2024-09-21T23:09:48.068366+0000","last_active":"2024-09-21T23:09:48.067730+0000","last_peered":"2024-09-21T23:09:48.067730+0000","last_clean":"2024-09-21T23:09:48.067730+0000","last_became_active":"2024-09-21T23:09:45.718621+0000","last_became_peered":"2024-09-21T23:09:45.718621+0000","last_unstale":"2024-09-21T23:09:48.067730+0000","last_undegraded":"2024-09-21T23:09:48.067730+0000","last_fullsized":"2024-09-21T23:09:48.067730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:44:48.782244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725647+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:09:47.725647+0000","last_peered":"2024-09-21T23:09:47.725647+0000","last_clean":"2024-09-21T23:09:47.725647+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:09:47.725647+0000","last_undegraded":"2024-09-21T23:09:47.725647+0000","last_fullsized":"2024-09-21T23:09:47.725647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068127+0000","last_change":"2024-09-21T23:09:48.068555+0000","last_active":"2024-09-21T23:09:48.068127+0000","last_peered":"2024-09-21T23:09:48.068127+0000","last_clean":"2024-09-21T23:09:48.068127+0000","last_became_active":"2024-09-21T23:09:45.869065+0000","last_became_peered":"2024-09-21T23:09:45.869065+0000","last_unstale":"2024-09-21T23:09:48.068127+0000","last_undegraded":"2024-09-21T23:09:48.068127+0000","last_fullsized":"2024-09-21T23:09:48.068127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:48.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088130999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725811+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:09:47.725811+0000","last_peered":"2024-09-21T23:09:47.725811+0000","last_clean":"2024-09-21T23:09:47.725811+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:09:47.725811+0000","last_undegraded":"2024-09-21T23:09:47.725811+0000","last_fullsized":"2024-09-21T23:09:47.725811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.724225+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:09:47.724225+0000","last_peered":"2024-09-21T23:09:47.724225+0000","last_clean":"2024-09-21T23:09:47.724225+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:09:47.724225+0000","last_undegraded":"2024-09-21T23:09:47.724225+0000","last_fullsized":"2024-09-21T23:09:47.724225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.716035+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:09:47.716035+0000","last_peered":"2024-09-21T23:09:47.716035+0000","last_clean":"2024-09-21T23:09:47.716035+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:09:47.716035+0000","last_undegraded":"2024-09-21T23:09:47.716035+0000","last_fullsized":"2024-09-21T23:09:47.716035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068015+0000","last_change":"2024-09-21T23:09:48.068542+0000","last_active":"2024-09-21T23:09:48.068015+0000","last_peered":"2024-09-21T23:09:48.068015+0000","last_clean":"2024-09-21T23:09:48.068015+0000","last_became_active":"2024-09-21T23:09:45.714516+0000","last_became_peered":"2024-09-21T23:09:45.714516+0000","last_unstale":"2024-09-21T23:09:48.068015+0000","last_undegraded":"2024-09-21T23:09:48.068015+0000","last_fullsized":"2024-09-21T23:09:48.068015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:51.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000999572,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.722583+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:09:47.722583+0000","last_peered":"2024-09-21T23:09:47.722583+0000","last_clean":"2024-09-21T23:09:47.722583+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:09:47.722583+0000","last_undegraded":"2024-09-21T23:09:47.722583+0000","last_fullsized":"2024-09-21T23:09:47.722583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067436+0000","last_change":"2024-09-21T23:09:41.675036+0000","last_active":"2024-09-21T23:09:48.067436+0000","last_peered":"2024-09-21T23:09:48.067436+0000","last_clean":"2024-09-21T23:09:48.067436+0000","last_became_active":"2024-09-21T23:09:41.674018+0000","last_became_peered":"2024-09-21T23:09:41.674018+0000","last_unstale":"2024-09-21T23:09:48.067436+0000","last_undegraded":"2024-09-21T23:09:48.067436+0000","last_fullsized":"2024-09-21T23:09:48.067436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:10.788395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:09:51.800 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:09:51.801 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T23:09:51.801 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T23:09:51.801 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T23:09:52.110 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:52.110 INFO:teuthology.orchestra.run.smithi150.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T23:09:52.127 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T23:09:52.127 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T23:09:52.138 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T23:09:52.138 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T23:09:52.138 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T23:09:52.139 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T23:09:52.437 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T23:09:52.437 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:09:52.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:09:52.708 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:52.708 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:09:52.725 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-21T23:09:52.488923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219008,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2834432,"data_stored":1731942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.786488"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067730+0000","last_change":"2024-09-21T23:09:48.068366+0000","last_active":"2024-09-21T23:09:48.067730+0000","last_peered":"2024-09-21T23:09:48.067730+0000","last_clean":"2024-09-21T23:09:48.067730+0000","last_became_active":"2024-09-21T23:09:45.718621+0000","last_became_peered":"2024-09-21T23:09:45.718621+0000","last_unstale":"2024-09-21T23:09:48.067730+0000","last_undegraded":"2024-09-21T23:09:48.067730+0000","last_fullsized":"2024-09-21T23:09:48.067730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:44:48.782244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725647+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:09:47.725647+0000","last_peered":"2024-09-21T23:09:47.725647+0000","last_clean":"2024-09-21T23:09:47.725647+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:09:47.725647+0000","last_undegraded":"2024-09-21T23:09:47.725647+0000","last_fullsized":"2024-09-21T23:09:47.725647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068127+0000","last_change":"2024-09-21T23:09:48.068555+0000","last_active":"2024-09-21T23:09:48.068127+0000","last_peered":"2024-09-21T23:09:48.068127+0000","last_clean":"2024-09-21T23:09:48.068127+0000","last_became_active":"2024-09-21T23:09:45.869065+0000","last_became_peered":"2024-09-21T23:09:45.869065+0000","last_unstale":"2024-09-21T23:09:48.068127+0000","last_undegraded":"2024-09-21T23:09:48.068127+0000","last_fullsized":"2024-09-21T23:09:48.068127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:48.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088130999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725811+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:09:47.725811+0000","last_peered":"2024-09-21T23:09:47.725811+0000","last_clean":"2024-09-21T23:09:47.725811+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:09:47.725811+0000","last_undegraded":"2024-09-21T23:09:47.725811+0000","last_fullsized":"2024-09-21T23:09:47.725811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.724225+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:09:47.724225+0000","last_peered":"2024-09-21T23:09:47.724225+0000","last_clean":"2024-09-21T23:09:47.724225+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:09:47.724225+0000","last_undegraded":"2024-09-21T23:09:47.724225+0000","last_fullsized":"2024-09-21T23:09:47.724225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.716035+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:09:47.716035+0000","last_peered":"2024-09-21T23:09:47.716035+0000","last_clean":"2024-09-21T23:09:47.716035+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:09:47.716035+0000","last_undegraded":"2024-09-21T23:09:47.716035+0000","last_fullsized":"2024-09-21T23:09:47.716035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068015+0000","last_change":"2024-09-21T23:09:48.068542+0000","last_active":"2024-09-21T23:09:48.068015+0000","last_peered":"2024-09-21T23:09:48.068015+0000","last_clean":"2024-09-21T23:09:48.068015+0000","last_became_active":"2024-09-21T23:09:45.714516+0000","last_became_peered":"2024-09-21T23:09:45.714516+0000","last_unstale":"2024-09-21T23:09:48.068015+0000","last_undegraded":"2024-09-21T23:09:48.068015+0000","last_fullsized":"2024-09-21T23:09:48.068015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:51.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000999572,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.722583+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:09:47.722583+0000","last_peered":"2024-09-21T23:09:47.722583+0000","last_clean":"2024-09-21T23:09:47.722583+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:09:47.722583+0000","last_undegraded":"2024-09-21T23:09:47.722583+0000","last_fullsized":"2024-09-21T23:09:47.722583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067436+0000","last_change":"2024-09-21T23:09:41.675036+0000","last_active":"2024-09-21T23:09:48.067436+0000","last_peered":"2024-09-21T23:09:48.067436+0000","last_clean":"2024-09-21T23:09:48.067436+0000","last_became_active":"2024-09-21T23:09:41.674018+0000","last_became_peered":"2024-09-21T23:09:41.674018+0000","last_unstale":"2024-09-21T23:09:48.067436+0000","last_undegraded":"2024-09-21T23:09:48.067436+0000","last_fullsized":"2024-09-21T23:09:48.067436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:10.788395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:09:52.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:09:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:53.001 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:09:53.017 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-21T23:09:52.488923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219008,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2834432,"data_stored":1731942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.786488"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067730+0000","last_change":"2024-09-21T23:09:48.068366+0000","last_active":"2024-09-21T23:09:48.067730+0000","last_peered":"2024-09-21T23:09:48.067730+0000","last_clean":"2024-09-21T23:09:48.067730+0000","last_became_active":"2024-09-21T23:09:45.718621+0000","last_became_peered":"2024-09-21T23:09:45.718621+0000","last_unstale":"2024-09-21T23:09:48.067730+0000","last_undegraded":"2024-09-21T23:09:48.067730+0000","last_fullsized":"2024-09-21T23:09:48.067730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:44:48.782244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725647+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:09:47.725647+0000","last_peered":"2024-09-21T23:09:47.725647+0000","last_clean":"2024-09-21T23:09:47.725647+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:09:47.725647+0000","last_undegraded":"2024-09-21T23:09:47.725647+0000","last_fullsized":"2024-09-21T23:09:47.725647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068127+0000","last_change":"2024-09-21T23:09:48.068555+0000","last_active":"2024-09-21T23:09:48.068127+0000","last_peered":"2024-09-21T23:09:48.068127+0000","last_clean":"2024-09-21T23:09:48.068127+0000","last_became_active":"2024-09-21T23:09:45.869065+0000","last_became_peered":"2024-09-21T23:09:45.869065+0000","last_unstale":"2024-09-21T23:09:48.068127+0000","last_undegraded":"2024-09-21T23:09:48.068127+0000","last_fullsized":"2024-09-21T23:09:48.068127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:48.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088130999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.725811+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:09:47.725811+0000","last_peered":"2024-09-21T23:09:47.725811+0000","last_clean":"2024-09-21T23:09:47.725811+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:09:47.725811+0000","last_undegraded":"2024-09-21T23:09:47.725811+0000","last_fullsized":"2024-09-21T23:09:47.725811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.724225+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:09:47.724225+0000","last_peered":"2024-09-21T23:09:47.724225+0000","last_clean":"2024-09-21T23:09:47.724225+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:09:47.724225+0000","last_undegraded":"2024-09-21T23:09:47.724225+0000","last_fullsized":"2024-09-21T23:09:47.724225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.716035+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:09:47.716035+0000","last_peered":"2024-09-21T23:09:47.716035+0000","last_clean":"2024-09-21T23:09:47.716035+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:09:47.716035+0000","last_undegraded":"2024-09-21T23:09:47.716035+0000","last_fullsized":"2024-09-21T23:09:47.716035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.068015+0000","last_change":"2024-09-21T23:09:48.068542+0000","last_active":"2024-09-21T23:09:48.068015+0000","last_peered":"2024-09-21T23:09:48.068015+0000","last_clean":"2024-09-21T23:09:48.068015+0000","last_became_active":"2024-09-21T23:09:45.714516+0000","last_became_peered":"2024-09-21T23:09:45.714516+0000","last_unstale":"2024-09-21T23:09:48.068015+0000","last_undegraded":"2024-09-21T23:09:48.068015+0000","last_fullsized":"2024-09-21T23:09:48.068015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:51.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000999572,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:47.722583+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:09:47.722583+0000","last_peered":"2024-09-21T23:09:47.722583+0000","last_clean":"2024-09-21T23:09:47.722583+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:09:47.722583+0000","last_undegraded":"2024-09-21T23:09:47.722583+0000","last_fullsized":"2024-09-21T23:09:47.722583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:09:48.067436+0000","last_change":"2024-09-21T23:09:41.675036+0000","last_active":"2024-09-21T23:09:48.067436+0000","last_peered":"2024-09-21T23:09:48.067436+0000","last_clean":"2024-09-21T23:09:48.067436+0000","last_became_active":"2024-09-21T23:09:41.674018+0000","last_became_peered":"2024-09-21T23:09:41.674018+0000","last_unstale":"2024-09-21T23:09:48.067436+0000","last_undegraded":"2024-09-21T23:09:48.067436+0000","last_fullsized":"2024-09-21T23:09:48.067436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:10.788395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":68896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":798720,"data_stored":659264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:09:53.018 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:09:53.018 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T23:09:53.288 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 20 2024-09-21T23:09:53.288 INFO:teuthology.orchestra.run.smithi150.stdout:fsid 9ce7cfc1-a7d6-426f-bf44-a308fb883b8f 2024-09-21T23:09:53.288 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-09-21T23:09:25.662940+0000 2024-09-21T23:09:53.288 INFO:teuthology.orchestra.run.smithi150.stdout:modified 2024-09-21T23:09:47.708037+0000 2024-09-21T23:09:53.288 INFO:teuthology.orchestra.run.smithi150.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T23:09:53.288 INFO:teuthology.orchestra.run.smithi150.stdout:crush_version 4 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:full_ratio 0.95 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:backfillfull_ratio 0.9 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:nearfull_ratio 0.85 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:require_min_compat_client luminous 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:min_compat_client jewel 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:require_osd_release squid 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:stretch_mode_enabled false 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 20 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: removed_snaps_queue [2~1] 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:max_osd 8 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/198356244,v1:172.21.15.150:6817/198356244] [v2:172.21.15.150:6818/198356244,v1:172.21.15.150:6819/198356244] exists,up 3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d 2024-09-21T23:09:53.289 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/725319294,v1:172.21.15.150:6825/725319294] [v2:172.21.15.150:6826/725319294,v1:172.21.15.150:6827/725319294] exists,up 5fb69564-3b15-470e-b45f-f70594751344 2024-09-21T23:09:53.290 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/1956153436,v1:172.21.15.150:6801/1956153436] [v2:172.21.15.150:6802/1956153436,v1:172.21.15.150:6803/1956153436] exists,up cc3e3e88-a324-4e3d-8b40-0d8c93da1400 2024-09-21T23:09:53.290 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/3049870482,v1:172.21.15.150:6809/3049870482] [v2:172.21.15.150:6810/3049870482,v1:172.21.15.150:6811/3049870482] exists,up 933fe042-8aed-4795-918c-572cf81999a7 2024-09-21T23:09:53.290 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/4112998952,v1:172.21.15.175:6801/4112998952] [v2:172.21.15.175:6802/4112998952,v1:172.21.15.175:6803/4112998952] exists,up af826961-2dad-4702-a904-991e2f5f57c8 2024-09-21T23:09:53.290 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6826/120048378,v1:172.21.15.175:6827/120048378] [v2:172.21.15.175:6828/120048378,v1:172.21.15.175:6829/120048378] exists,up c4f1908b-0885-408e-873e-4d2aa9b9c506 2024-09-21T23:09:53.290 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6818/787648514,v1:172.21.15.175:6819/787648514] [v2:172.21.15.175:6820/787648514,v1:172.21.15.175:6821/787648514] exists,up 08cef493-ce38-4b52-b5f2-23e7694953c0 2024-09-21T23:09:53.290 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/1424462696,v1:172.21.15.175:6809/1424462696] [v2:172.21.15.175:6810/1424462696,v1:172.21.15.175:6811/1424462696] exists,up 26d2142e-419c-4da3-9c28-ac442b1b66fb 2024-09-21T23:09:53.305 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/198356244,v1:172.21.15.150:6817/198356244] [v2:172.21.15.150:6818/198356244,v1:172.21.15.150:6819/198356244] exists,up 3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d', 'osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/725319294,v1:172.21.15.150:6825/725319294] [v2:172.21.15.150:6826/725319294,v1:172.21.15.150:6827/725319294] exists,up 5fb69564-3b15-470e-b45f-f70594751344', 'osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/1956153436,v1:172.21.15.150:6801/1956153436] [v2:172.21.15.150:6802/1956153436,v1:172.21.15.150:6803/1956153436] exists,up cc3e3e88-a324-4e3d-8b40-0d8c93da1400', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/3049870482,v1:172.21.15.150:6809/3049870482] [v2:172.21.15.150:6810/3049870482,v1:172.21.15.150:6811/3049870482] exists,up 933fe042-8aed-4795-918c-572cf81999a7', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/4112998952,v1:172.21.15.175:6801/4112998952] [v2:172.21.15.175:6802/4112998952,v1:172.21.15.175:6803/4112998952] exists,up af826961-2dad-4702-a904-991e2f5f57c8', 'osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6826/120048378,v1:172.21.15.175:6827/120048378] [v2:172.21.15.175:6828/120048378,v1:172.21.15.175:6829/120048378] exists,up c4f1908b-0885-408e-873e-4d2aa9b9c506', 'osd.6 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6818/787648514,v1:172.21.15.175:6819/787648514] [v2:172.21.15.175:6820/787648514,v1:172.21.15.175:6821/787648514] exists,up 08cef493-ce38-4b52-b5f2-23e7694953c0', 'osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/1424462696,v1:172.21.15.175:6809/1424462696] [v2:172.21.15.175:6810/1424462696,v1:172.21.15.175:6811/1424462696] exists,up 26d2142e-419c-4da3-9c28-ac442b1b66fb'] 2024-09-21T23:09:53.306 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mon.a", 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster": "ceph", 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "admin_socket_mode": "", 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_client_required": "cephx, none", 2024-09-21T23:09:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_cluster_required": "cephx", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_debug": "true", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_service_required": "cephx", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "auth_supported": "cephx", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_aio": "true", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T23:09:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_aio_reap_max": "16", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_async_discard": "false", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_block_size": "4096", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_debug_aio": "false", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_enable_discard": "false", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_flock_retry": "3", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T23:09:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_inject_crash": "0", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_ioring": "false", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_ioring_hipri": "false", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bdev_type": "", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_allocator": "hybrid", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_buffered_io": "true", 2024-09-21T23:09:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_replay_recovery": "false", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T23:09:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_sync_write": "false", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_allocator": "hybrid", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bdev_label_multi": "true", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T23:09:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_create": "true", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_db_create": "false", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_db_path": "", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_db_size": "0", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_path": "", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_size": "107374182400", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_wal_create": "false", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_wal_path": "", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bluefs": "true", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T23:09:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_autotune": "true", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_size": "0", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T23:09:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_type": "2q", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_clone_cow": "true", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_mode": "none", 2024-09-21T23:09:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_freelist": "false", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T23:09:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T23:09:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fail_eio": "false", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T23:09:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T23:09:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_max_blob_size": "0", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T23:09:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T23:09:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_spdk_mem": "512", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T23:09:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_tracing": "false", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T23:09:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephx_cluster_require_version": "2", 2024-09-21T23:09:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "cephx_require_signatures": "false", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "cephx_require_version": "2", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "cephx_service_require_signatures": "false", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "cephx_service_require_version": "2", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "cephx_sign_messages": "true", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "chdir": "", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_acl_type": "", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_asio_thread_count": "2", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_cache_mid": "0.750000", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_cache_size": "16384", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_caps_release_delay": "5", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_check_pool_perm": "true", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_debug_force_sync_read": "false", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_debug_getattr_caps": "false", 2024-09-21T23:09:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_debug_inject_features": "", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_die_on_failed_remount": "false", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_dirsize_rbytes": "true", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_force_lazyio": "false", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_fs": "", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_inject_release_failure": "false", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_max_inline_size": "4096", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_mds_namespace": "", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_metadata": "", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_mount_gid": "-1", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_mount_timeout": "300", 2024-09-21T23:09:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_mount_uid": "-1", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_mountpoint": "/", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_notify_timeout": "10", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_oc": "true", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_oc_max_objects": "1000", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_oc_size": "209715200", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_permissions": "true", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_quota": "true", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_quota_df": "true", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_readahead_max_bytes": "0", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_readahead_max_periods": "4", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_readahead_min": "131072", 2024-09-21T23:09:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_reconnect_stale": "false", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_shutdown_timeout": "30", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_snapdir": ".snap", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_tick_interval": "1", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_trace": "", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_use_faked_inos": "false", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_use_random_mds": "false", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "clog_to_graylog": "false", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "clog_to_graylog_port": "12201", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "clog_to_monitors": "default=true", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "clog_to_syslog": "false", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T23:09:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "clog_to_syslog_level": "info", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "-", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_network": "", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_network_interface": "", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "compressor_zlib_isal": "false", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "compressor_zlib_level": "5", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "compressor_zstd_level": "1", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T23:09:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crimson_osd_stat_interval": "0", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crimson_seastar_cpu_cores": "", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crimson_seastar_num_threads": "0", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_location": "", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_location_hook": "", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_location_hook_timeout": "10", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "daemonize": "false", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "daos_pool": "tank", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T23:09:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_alienstore": "0/5", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_asok": "1/5", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_asok_assert_abort": "false", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_auth": "20/20", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_bdev": "1/3", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_bluefs": "1/5", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_bluestore": "1/5", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_buffer": "0/1", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_cephsqlite": "0/5", 2024-09-21T23:09:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_client": "0/5", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_compressor": "1/5", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_context": "0/1", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_crimson_interrupt": "0/5", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_crush": "1/1", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_crypto": "1/5", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_cyanstore": "0/5", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_dpdk": "1/5", 2024-09-21T23:09:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_eventtrace": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_filer": "0/1", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_filestore": "1/3", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_finisher": "1/1", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_fuse": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_javaclient": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_journal": "1/3", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_journaler": "0/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_kstore": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_lockdep": "0/1", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mclock": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mds": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mds_balancer": "1/5", 2024-09-21T23:09:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mds_locker": "1/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mds_log": "1/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mds_migrator": "1/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mds_quiesce": "3/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_memstore": "1/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mgr": "2/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mgrc": "1/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_mon": "20/20", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_monc": "0/10", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_ms": "1/1", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_none": "0/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_objclass": "0/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_objectcacher": "0/5", 2024-09-21T23:09:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_objecter": "0/1", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_optracker": "0/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_osd": "1/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_paxos": "20/20", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_perfcounter": "1/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_prioritycache": "1/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rados": "0/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rbd": "0/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rbd_replay": "0/5", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_refs": "0/0", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_reserver": "1/1", 2024-09-21T23:09:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rgw": "1/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rgw_access": "1/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rgw_flight": "1/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rgw_sync": "1/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_rocksdb": "4/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore": "0/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_backref": "0/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_cache": "0/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_device": "0/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_epm": "0/5", 2024-09-21T23:09:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_journal": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_lba": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_odata": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_omap": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_onode": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_t": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_seastore_tm": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_striper": "0/1", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_test": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_throttle": "1/1", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_timer": "0/1", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_tp": "0/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "debug_trace": "1/5", 2024-09-21T23:09:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "device_failure_prediction_mode": "none", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "err_to_graylog": "false", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "err_to_journald": "false", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "err_to_stderr": "true", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "err_to_syslog": "false", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event_tracing": "false", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "exporter_http_port": "9926", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "exporter_prio_limit": "5", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "exporter_sort_metrics": "true", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "exporter_stats_period": "5", 2024-09-21T23:09:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "fake_statfs_for_testing": "0", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "fatal_signal_handlers": "true", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filer_max_purge_ops": "10", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filer_max_truncate_ops": "128", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_blackhole": "false", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_btrfs_snap": "true", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_caller_concurrency": "10", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T23:09:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_debug_omap_check": "false", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_debug_verify_split": "false", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_dump_file": "", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_fadvise": "true", 2024-09-21T23:09:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_fail_eio": "true", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_fd_cache_size": "128", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_fiemap": "false", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_inject_stall": "0", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_journal_parallel": "false", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_journal_trailing": "false", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_journal_writeahead": "false", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_kill_at": "0", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T23:09:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_merge_threshold": "-10", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_odsync_write": "false", 2024-09-21T23:09:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_omap_backend_path": "", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_op_threads": "2", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_punch_hole": "false", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T23:09:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_queue_max_ops": "50", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_seek_data_hole": "false", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_sloppy_crc": "false", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_splice": "false", 2024-09-21T23:09:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_split_multiple": "2", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_split_rand_factor": "20", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_update_to": "1000", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T23:09:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_xfs_extsize": "false", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "filestore_zfs_snap": "false", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fio_dir": "/tmp/fio", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fsid": "9ce7cfc1-a7d6-426f-bf44-a308fb883b8f", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_allow_other": "true", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_big_writes": "true", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_debug": "false", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_default_permissions": "false", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_disable_pagecache": "false", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_max_write": "0", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_multithreaded": "true", 2024-09-21T23:09:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_require_active_mds": "true", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_set_user_groups": "true", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_splice_move": "true", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_splice_read": "true", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_splice_write": "true", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "gss_target_name": "ceph", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_file": "", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_inject_failure": "0", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_interval": "5", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "host": "smithi150", 2024-09-21T23:09:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T23:09:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "inject_early_sigterm": "false", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "jaeger_agent_port": "6799", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "jaeger_tracing_enable": "false", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_aio": "true", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_align_min_size": "65536", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_block_align": "true", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_block_size": "4096", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_dio": "true", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_discard": "false", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_force_aio": "false", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_ignore_corruption": "false", 2024-09-21T23:09:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_max_write_entries": "100", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_replay_from": "0", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_write_header_frequency": "0", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journal_zero_on_create": "false", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journaler_prefetch_periods": "10", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journaler_prezero_periods": "5", 2024-09-21T23:09:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "journaler_write_head_interval": "15", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "key": "", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "keyfile": "", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_backend": "rocksdb", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_max_bytes": "67108864", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_max_ops": "512", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_onode_map_size": "1024", 2024-09-21T23:09:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "kstore_sync_transaction": "false", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "librados_thread_count": "2", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "lockdep": "false", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "lockdep_force_backtrace": "false", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_coarse_timestamps": "true", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_flush_on_exit": "false", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_graylog_port": "12201", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_max_new": "1000", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_max_recent": "10000", 2024-09-21T23:09:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_stderr_prefix": "", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": "true", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_graylog": "false", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_journald": "false", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_stderr": "false", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_syslog": "false", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_action_on_write_error": "1", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_alternate_name_max": "8192", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_asio_thread_count": "2", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_export_pin": "true", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T23:09:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_interval": "10", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_max": "-1", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_max_until": "-1", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_merge_size": "50", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_mode": "0", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_overload_epochs": "2", 2024-09-21T23:09:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_split_bits": "3", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_split_size": "10000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_mid": "0.700000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-21T23:09:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_quiesce_delay": "0", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_trim_interval": "1", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T23:09:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_debug_auth_pins": "false", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_debug_frag": "false", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_debug_scatterstat": "false", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_debug_subtrees": "false", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_default_dir_hash": "2", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_defer_session_stale": "true", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dir_max_entries": "0", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dir_prefetch": "true", 2024-09-21T23:09:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_early_reply": "true", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_enable_op_tracker": "true", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_enforce_unique_name": "true", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T23:09:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_inject_health_dummy": "false", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T23:09:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_join_fs": "", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_journal_format": "1", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_create_at": "0", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_export_at": "0", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_import_at": "0", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_journal_at": "0", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T23:09:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_link_at": "0", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_openc_at": "0", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_rename_at": "0", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_kill_shutdown_at": "0", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_event_large_threshold": "524288", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_max_events": "-1", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_max_segments": "128", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_pause": "false", 2024-09-21T23:09:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_segment_size": "0", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_trim_threshold": "128", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_completed_requests": "100000", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_export_size": "20971520", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_file_recover": "32", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_purge_files": "64", 2024-09-21T23:09:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_purge_ops": "8192", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_metrics_update_interval": "2", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_min_caps_per_client": "100", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_numa_node": "-1", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_op_history_duration": "600", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_op_history_size": "20", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T23:09:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_op_log_threshold": "5", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_ping_grace": "15", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_ping_interval": "5", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_recall_max_caps": "30000", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_replay_interval": "1.000000", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_root_ino_gid": "0", 2024-09-21T23:09:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_root_ino_uid": "0", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_shutdown_check": "0", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_skip_ino": "0", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T23:09:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_snap_min_uid": "0", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_snap_rstat": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_symlink_recovery": "true", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_thrash_exports": "0", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_thrash_fragments": "0", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_tick_interval": "5.000000", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_valgrind_exit": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_verify_backtrace": "1", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_verify_scatter": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_wipe_sessions": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool_debug": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T23:09:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "memstore_page_set": "false", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "memstore_page_size": "65536", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_client_bytes": "134217728", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_client_messages": "512", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_disabled_modules": "", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_enable_op_tracker": "true", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_mds_messages": "128", 2024-09-21T23:09:53.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_mon_messages": "128", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_op_history_duration": "600", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_op_history_size": "20", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_op_log_threshold": "5", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_osd_messages": "8192", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_pool": "true", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T23:09:53.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_standby_modules": "true", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_stats_period": "5", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_stats_threshold": "5", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_tick_period": "2", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_allow_pool_delete": "true", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_bytes": "104857600", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T23:09:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_client_target_rank": "-1", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T23:09:53.481 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_compact_on_start": "false", 2024-09-21T23:09:53.482 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_compact_on_trim": "true", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_cpu_threads": "4", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_data_avail_crit": "5", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_data_avail_warn": "5", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T23:09:53.483 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_dump_json": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_extra_checks": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_no_require_squid": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_down_uptime_grace": "60", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_election_default_strategy": "1", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_election_timeout": "5.000000", 2024-09-21T23:09:53.484 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_enable_op_tracker": "true", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_fake_pool_delete": "false", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_force_quorum_join": "false", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_health_log_update_period": "5", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_health_max_detail": "50", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_health_to_clog": "true", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_host": "172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790],172.21.15.175", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_host_override": "", 2024-09-21T23:09:53.485 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_initial_members": "", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_lease": "5.000000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_log_full_interval": "50", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_log_max": "10000", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_log_max_summary": "50", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T23:09:53.486 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_log_epochs": "500", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_osd": "10000", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_memory_autotune": "true", 2024-09-21T23:09:53.487 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_memory_target": "2147483648", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mgr_digest_period": "5", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_op_complaint_time": "30", 2024-09-21T23:09:53.488 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_op_history_duration": "600", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_op_history_size": "20", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_op_log_threshold": "5", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T23:09:53.489 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_cache_size": "500", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T23:09:53.490 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T23:09:53.491 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T23:09:53.492 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_report_timeout": "900", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T23:09:53.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T23:09:53.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_reweight_max_osds": "4", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T23:09:53.495 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_scrub_interval": "86400", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_scrub_max_keys": "100", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_scrub_timeout": "300", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_session_timeout": "300", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_smart_report_timeout": "5", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_stretch_pool_size": "4", 2024-09-21T23:09:53.496 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_sync_debug": "false", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T23:09:53.497 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_tick_interval": "5", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T23:09:53.498 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_older_version": "true", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T23:09:53.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "monmap": "", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T23:09:53.500 INFO:teuthology.orchestra.run.smithi150.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "motr_tracing_enabled": "false", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_op_threads": "3", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T23:09:53.501 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_cm": "false", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_device_name": "", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T23:09:53.502 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_sl": "3", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_rdma_type": "ib", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_async_reap_threshold": "5", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_before_connect": "false", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_ipv4": "true", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_ipv6": "false", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_msgr1": "true", 2024-09-21T23:09:53.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_msgr2": "true", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_port_max": "7568", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_port_min": "6800", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_retry_count": "3", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_bind_retry_delay": "5", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_blackhole_client": "false", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_blackhole_mds": "false", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_blackhole_mgr": "false", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_blackhole_mon": "false", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_blackhole_osd": "false", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_client_mode": "crc secure", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_cluster_type": "", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_compress_secure": "false", 2024-09-21T23:09:53.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_crc_data": "true", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_crc_header": "true", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_die_on_bug": "true", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_die_on_old_message": "true", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T23:09:53.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_hugepages": "", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_lro": "true", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_pmd": "", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_port_id": "0", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_dump_on_send": "false", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T23:09:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_inject_delay_type": "", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_inject_network_congestion": "0", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_max_accept_failures": "4", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_max_backoff": "15.000000", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_osd_compress_mode": "none", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T23:09:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_pq_min_cost": "65536", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_public_type": "", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_service_mode": "crc secure", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_tcp_nodelay": "true", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "ms_type": "async+posix", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "no_config_file": "false", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_inflight_ops": "1024", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T23:09:53.508 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter_timeout": "10.000000", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "objectstore_blackhole": "false", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "openssl_engine_opts": "", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_agent_max_ops": "4", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_agent_slop": "0.020000", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_asio_thread_count": "2", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T23:09:53.509 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_backfill_scan_max": "512", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_backfill_scan_min": "64", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_backoff_on_peering": "false", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_beacon_report_interval": "300", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bench_duration": "30", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_blkin_trace_all": "false", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T23:09:53.510 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_class_update_on_start": "true", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_client_message_cap": "256", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_client_op_priority": "63", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_client_watch_timeout": "30", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_command_max_records": "256", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_compact_on_start": "false", 2024-09-21T23:09:53.511 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_crush_update_on_start": "true", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T23:09:53.512 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_op_order": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_shutdown": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T23:09:53.513 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_trim_objects": "false", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T23:09:53.514 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_default_notify_timeout": "30", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_enable_op_tracker": "true", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_fast_info": "true", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_fast_shutdown": "true", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T23:09:53.515 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_function_tracing": "false", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_heartbeat_grace": "20", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_heartbeat_interval": "6", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_heartbeat_stale": "600", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T23:09:53.516 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_journal_size": "5120", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_kill_backfill_at": "0", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_map_cache_size": "50", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_map_dedup": "true", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_map_message_max": "40", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T23:09:53.517 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_attr_name_len": "100", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_attr_size": "0", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_backfills": "1", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_markdown_count": "5", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_markdown_period": "600", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_object_name_len": "2048", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_object_size": "134217728", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_pgls": "1024", 2024-09-21T23:09:53.518 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_push_cost": "8388608", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_push_objects": "10", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_scrubs": "3", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_max_write_size": "90", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T23:09:53.519 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_profile": "balanced", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_memory_base": "805306368", 2024-09-21T23:09:53.520 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_memory_target": "4294967296", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_memory_target_autotune": "false", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_min_recovery_priority": "0", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mon_report_interval": "5", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T23:09:53.521 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_num_cache_shards": "32", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_numa_node": "-1", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_objecter_finishers": "1", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_objectstore": "bluestore", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_objectstore_fuse": "false", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_objectstore_tracing": "false", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_history_duration": "600", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_history_size": "20", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T23:09:53.522 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_log_threshold": "5", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_num_shards": "0", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_op_thread_timeout": "15", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_open_classes_on_start": "true", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_os_flags": "0", 2024-09-21T23:09:53.523 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_peering_op_priority": "255", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_delete_priority": "5", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T23:09:53.524 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_crimson": "false", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_flags": "0", 2024-09-21T23:09:53.525 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_min_size": "0", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_read_ratio": "70", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_size": "2", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_default_type": "replicated", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T23:09:53.526 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_cost": "20971520", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_max_active": "0", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_op_priority": "3", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_priority": "5", 2024-09-21T23:09:53.527 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_repair_during_recovery": "false", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T23:09:53.528 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_cost": "52428800", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_end_hour": "0", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T23:09:53.529 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_priority": "5", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_skip_data_digest": "false", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_smart_report_timeout": "5", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_snap_trim_priority": "5", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T23:09:53.530 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_target_transaction_size": "30", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_tracing": "false", 2024-09-21T23:09:53.531 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_use_stale_snap": "false", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_kill_at": "0", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_max_join_drift": "10", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_min": "500", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_min_wait": "0.050000", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_service_trim_max": "500", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_service_trim_min": "250", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_stash_full_interval": "25", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_trim_max": "500", 2024-09-21T23:09:53.532 INFO:teuthology.orchestra.run.smithi150.stdout: "paxos_trim_min": "250", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "perf": "true", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "-", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrv": "", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "public_bind_addr": "-", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "public_network": "", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "public_network_interface": "", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "qat_compressor_busy_polling": "false", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "qat_compressor_enabled": "false", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T23:09:53.533 INFO:teuthology.orchestra.run.smithi150.stdout: "rados_mon_op_timeout": "0", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rados_osd_op_timeout": "0", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rados_tracing": "false", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_atime_update_interval": "60", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache": "true", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T23:09:53.534 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_policy": "writearound", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_size": "33554432", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_compression_hint": "none", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_clone_format": "auto", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_data_pool": "", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_features": "61", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_format": "2", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_map_options": "", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_order": "22", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_pool": "rbd", 2024-09-21T23:09:53.535 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_stripe_count": "0", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_io_scheduler": "simple", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T23:09:53.536 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_order": "24", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_pool": "", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_journal_splay_width": "4", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T23:09:53.537 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T23:09:53.538 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_op_threads": "1", 2024-09-21T23:09:53.539 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_plugins": "", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T23:09:53.540 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T23:09:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_tracing": "false", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_validate_names": "true", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_validate_pool": "true", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "restapi_base_url": "", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "restapi_log_level": "", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_admin_entry": "admin", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_backend_store": "rados", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_barbican_url": "", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_beast_enable_async": "true", 2024-09-21T23:09:53.542 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_counters_cache": "false", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_cache_enabled": "true", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_cache_lru_size": "25000", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_config_store": "rados", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_content_length_compat": "false", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T23:09:53.543 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_cross_domain_policy": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T23:09:53.544 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T23:09:53.545 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T23:09:53.546 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_d4n_port": "6379", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T23:09:53.547 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data_log_window": "30", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T23:09:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dns_name": "", 2024-09-21T23:09:53.549 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dns_s3website_name": "", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enable_ops_log": "false", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enable_static_website": "false", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enable_usage_log": "false", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_expose_bucket": "false", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_extended_http_attrs": "", 2024-09-21T23:09:53.550 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_filter": "none", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_max_objs": "32", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T23:09:53.551 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_init_timeout": "300", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_kafka_connection_idle": "300", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T23:09:53.552 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_password": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_project": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_token": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_admin_user": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_api_version": "2", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T23:09:53.553 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_url": "", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lc_max_objs": "32", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lc_max_worker": "3", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T23:09:53.554 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ldap_uri": "", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_log_http_headers": "", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-21T23:09:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_attr_size": "0", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_listing_results": "1000", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_notify_retries": "10", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T23:09:53.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T23:09:53.557 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_num_control_oids": "8", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_numa_node": "-1", 2024-09-21T23:09:53.558 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_op_tracing": "false", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_opa_token": "", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_opa_url": "", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ops_log_rados": "false", 2024-09-21T23:09:53.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_posix_cache_lanes": "3", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_posix_cache_partitions": "3", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-21T23:09:53.560 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_print_continue": "true", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_rados_tracing": "false", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_realm": "", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_realm_id": "", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_region": "", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T23:09:53.561 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_request_uri": "", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_resolve_cname": "false", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_run_sync_thread": "true", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T23:09:53.562 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_script_uri": "", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_service_provider_name": "", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sts_client_id": "", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sts_client_secret": "", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sts_entry": "sts", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sts_key": "sts", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T23:09:53.563 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_auth_url": "", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_custom_header": "", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_need_stats": "true", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_tenant_name": "", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_url": "", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_lease_period": "120", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T23:09:53.564 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_thread_pool_size": "512", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_comment": "", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_createby": "", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_encoding": "", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_flag": "false", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_origin": "", 2024-09-21T23:09:53.565 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_torrent_tracker": "", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_usage_max_shards": "32", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_use_opa_authz": "false", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_counters_cache": "false", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T23:09:53.566 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_user_unique_email": "true", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_verify_ssl": "true", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_zone": "", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_zone_id": "", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_zonegroup": "", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_zonegroup_id": "", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T23:09:53.567 INFO:teuthology.orchestra.run.smithi150.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_block_size": "4096", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T23:09:53.568 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_partition_filters": "false", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_perf": "false", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "run_dir": "/var/run/ceph", 2024-09-21T23:09:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_block_create": "true", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_device_size": "53687091200", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_full_integrity_check": "false", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T23:09:53.570 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_main_device_type": "SSD", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T23:09:53.571 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "seastore_segment_size": "67108864", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "set_keepcaps": "false", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "setgroup": "", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "setuser": "", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "setuser_match_path": "", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "thp": "false", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "threadpool_default_timeout": "60", 2024-09-21T23:09:53.572 INFO:teuthology.orchestra.run.smithi150.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T23:09:53.573 INFO:teuthology.orchestra.run.smithi150.stdout: "throttler_perf_counter": "true" 2024-09-21T23:09:53.573 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:53.575 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T23:09:53.714 INFO:teuthology.orchestra.run.smithi150.stdout:mon.a: {} 2024-09-21T23:09:53.714 INFO:teuthology.orchestra.run.smithi150.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:09:53.726 INFO:teuthology.orchestra.run.smithi150.stdout:mon.b: {} 2024-09-21T23:09:53.726 INFO:teuthology.orchestra.run.smithi150.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:09:53.735 INFO:teuthology.orchestra.run.smithi150.stdout:mon.c: {} 2024-09-21T23:09:53.735 INFO:teuthology.orchestra.run.smithi150.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:09:53.750 DEBUG:teuthology.orchestra.run.smithi175:> type ceph-objectstore-tool 2024-09-21T23:09:53.756 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:09:53.756 DEBUG:teuthology.orchestra.run.smithi150:> type ceph-objectstore-tool 2024-09-21T23:09:53.760 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:09:53.760 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-21T23:09:53.776 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-21T23:09:53.777 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T23:09:53.777 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T23:09:53.777 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:09:53.777 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T23:09:53.777 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:09:53.777 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T23:09:53.778 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T23:09:53.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:09:53.778 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T23:09:53.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:09:53.779 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T23:09:53.779 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:09:53.779 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-21T23:09:53.808 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:53.806+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:53.830 INFO:teuthology.orchestra.run.smithi150.stdout:mapped writes DISABLED 2024-09-21T23:09:53.830 INFO:teuthology.orchestra.run.smithi150.stdout:mapped reads DISABLED 2024-09-21T23:09:53.830 INFO:teuthology.orchestra.run.smithi150.stdout:Seed set to 4238 2024-09-21T23:09:53.879 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:53.878+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:09:53.923 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:53.923 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:09:53.923 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:09:53.923 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:53.943 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:09:53.970 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:09:53.970 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:09:53.980 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:53.978+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:53.987 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:09:53.988 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:09:54.004 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:09:54.004 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:09:54.022 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:09:54.022 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:09:54.079 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:54.079 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:54.080 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:09:54.078+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:09:54.092 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:09:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:09:54.104 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "epoch": 20, 2024-09-21T23:09:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "fsid": "9ce7cfc1-a7d6-426f-bf44-a308fb883b8f", 2024-09-21T23:09:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "created": "2024-09-21T23:09:25.662940+0000", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "modified": "2024-09-21T23:09:47.708037+0000", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "last_up_change": "2024-09-21T23:09:39.531143+0000", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "last_in_change": "2024-09-21T23:09:28.692303+0000", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_num": 5799936, 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_set": [ 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "pglog_hardlimit", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "purged_snapdirs", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_deletes", 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "sortbitwise" 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_version": 4, 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:09:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_snapshot": "", 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_max": 2, 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd": 8, 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "min_compat_client": "jewel", 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "require_osd_release": "squid", 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_crimson": false, 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "pools": [ 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 1, 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": ".mgr", 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:40.569134+0000", 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-09-21T23:09:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool", 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 1, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 1, 2024-09-21T23:09:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 1, 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 1, 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 1, 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "16", 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:09:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 0, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 0, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:09:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:09:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "options": { 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_max": 32, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_min": 1 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": {} 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 8, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 8, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 0.25, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 2, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 2, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 2, 2024-09-21T23:09:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": "rbd", 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:44.626156+0000", 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 8193, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:09:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 8, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 8, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 8, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 8, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 8, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "20", 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 2, 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 20, 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:09:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "options": {}, 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd": {} 2024-09-21T23:09:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 1.9900000095367432, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 1.9900000095367432, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 1.75, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 1.75, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:09:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 0, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d", 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 0, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6816", 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6817", 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6818", 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6819", 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6822", 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6823", 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6820", 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6821", 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6817/198356244", 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6819/198356244", 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6823/198356244", 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6821/198356244", 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 1, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "5fb69564-3b15-470e-b45f-f70594751344", 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 17, 2024-09-21T23:09:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6824", 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6825", 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6826", 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6827", 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6830", 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6831", 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.127 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6828", 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6829", 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6825/725319294", 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6827/725319294", 2024-09-21T23:09:54.128 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6831/725319294", 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6829/725319294", 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 2, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "cc3e3e88-a324-4e3d-8b40-0d8c93da1400", 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.129 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 17, 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6800", 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6801", 2024-09-21T23:09:54.130 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6802", 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6803", 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6806", 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6807", 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6804", 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6805", 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6801/1956153436", 2024-09-21T23:09:54.133 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6803/1956153436", 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6807/1956153436", 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6805/1956153436", 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 3, 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "933fe042-8aed-4795-918c-572cf81999a7", 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.134 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 0, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6808", 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6809", 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6810", 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6811", 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6814", 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.137 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6815", 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6812", 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6813", 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6809/3049870482", 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6811/3049870482", 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6815/3049870482", 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6813/3049870482", 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.139 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 4, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "af826961-2dad-4702-a904-991e2f5f57c8", 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 0, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.140 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6800", 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6801", 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6802", 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6803", 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6806", 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6807", 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6804", 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6805", 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6801/4112998952", 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6803/4112998952", 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6807/4112998952", 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6805/4112998952", 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 5, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "c4f1908b-0885-408e-873e-4d2aa9b9c506", 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 17, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6826", 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6827", 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6828", 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6829", 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6832", 2024-09-21T23:09:54.147 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6833", 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6830", 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.148 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6831", 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6827/120048378", 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6829/120048378", 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6833/120048378", 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6831/120048378", 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.149 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 6, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "08cef493-ce38-4b52-b5f2-23e7694953c0", 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 17, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6818", 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6819", 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6820", 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6821", 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.152 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6824", 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6825", 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.153 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6822", 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6823", 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6819/787648514", 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6821/787648514", 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6825/787648514", 2024-09-21T23:09:54.154 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6823/787648514", 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 7, 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "26d2142e-419c-4da3-9c28-ac442b1b66fb", 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:09:54.155 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 17, 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6808", 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.156 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6809", 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6810", 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6811", 2024-09-21T23:09:54.157 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6814", 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6815", 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6812", 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:09:54.159 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6813", 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6809/1424462696", 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6811/1424462696", 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6815/1424462696", 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6813/1424462696", 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_xinfo": [ 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 0, 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:36.004700+0000", 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 1, 2024-09-21T23:09:54.161 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.958497+0000", 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 2, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.988302+0000", 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 3, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.993971+0000", 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 4, 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.163 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.945648+0000", 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 5, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:36.366302+0000", 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 6, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.852613+0000", 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 7, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:09:54.165 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.713697+0000", 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap": [], 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap_items": [], 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap_primaries": [], 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_temp": [], 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_temp": [], 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklist": {}, 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "range_blocklist": {}, 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profiles": { 2024-09-21T23:09:54.166 INFO:teuthology.orchestra.run.smithi150.stdout: "default": { 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-failure-domain": "osd", 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "k": "2", 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "m": "1", 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "plugin": "jerasure", 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "technique": "reed_sol_van" 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps_queue": [ 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 2, 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "snaps": [ 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "begin": 2, 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "length": 1 2024-09-21T23:09:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "new_removed_snaps": [ 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 2, 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "snaps": [ 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "begin": 2, 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "length": 1 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "new_purged_snaps": [], 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_node_flags": {}, 2024-09-21T23:09:54.168 INFO:teuthology.orchestra.run.smithi150.stdout: "device_class_flags": {}, 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode": { 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode_enabled": false, 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_bucket_count": 0, 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode_bucket": 0 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:54.169 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:54.171 INFO:tasks.thrashosds.thrasher:j is {'epoch': 20, 'fsid': '9ce7cfc1-a7d6-426f-bf44-a308fb883b8f', 'created': '2024-09-21T23:09:25.662940+0000', 'modified': '2024-09-21T23:09:47.708037+0000', 'last_up_change': '2024-09-21T23:09:39.531143+0000', 'last_in_change': '2024-09-21T23:09:28.692303+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:09:40.569134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:09:44.626156+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 198356244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 198356244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 198356244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 198356244}]}, 'public_addr': '172.21.15.150:6817/198356244', 'cluster_addr': '172.21.15.150:6819/198356244', 'heartbeat_back_addr': '172.21.15.150:6823/198356244', 'heartbeat_front_addr': '172.21.15.150:6821/198356244', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5fb69564-3b15-470e-b45f-f70594751344', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 725319294}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 725319294}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 725319294}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 725319294}]}, 'public_addr': '172.21.15.150:6825/725319294', 'cluster_addr': '172.21.15.150:6827/725319294', 'heartbeat_back_addr': '172.21.15.150:6831/725319294', 'heartbeat_front_addr': '172.21.15.150:6829/725319294', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'cc3e3e88-a324-4e3d-8b40-0d8c93da1400', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 1956153436}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 1956153436}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 1956153436}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 1956153436}]}, 'public_addr': '172.21.15.150:6801/1956153436', 'cluster_addr': '172.21.15.150:6803/1956153436', 'heartbeat_back_addr': '172.21.15.150:6807/1956153436', 'heartbeat_front_addr': '172.21.15.150:6805/1956153436', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '933fe042-8aed-4795-918c-572cf81999a7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 3049870482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 3049870482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 3049870482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 3049870482}]}, 'public_addr': '172.21.15.150:6809/3049870482', 'cluster_addr': '172.21.15.150:6811/3049870482', 'heartbeat_back_addr': '172.21.15.150:6815/3049870482', 'heartbeat_front_addr': '172.21.15.150:6813/3049870482', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'af826961-2dad-4702-a904-991e2f5f57c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6801', 'nonce': 4112998952}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6803', 'nonce': 4112998952}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6807', 'nonce': 4112998952}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6805', 'nonce': 4112998952}]}, 'public_addr': '172.21.15.175:6801/4112998952', 'cluster_addr': '172.21.15.175:6803/4112998952', 'heartbeat_back_addr': '172.21.15.175:6807/4112998952', 'heartbeat_front_addr': '172.21.15.175:6805/4112998952', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c4f1908b-0885-408e-873e-4d2aa9b9c506', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6826', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6827', 'nonce': 120048378}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6828', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6829', 'nonce': 120048378}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 120048378}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6830', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6831', 'nonce': 120048378}]}, 'public_addr': '172.21.15.175:6827/120048378', 'cluster_addr': '172.21.15.175:6829/120048378', 'heartbeat_back_addr': '172.21.15.175:6833/120048378', 'heartbeat_front_addr': '172.21.15.175:6831/120048378', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '08cef493-ce38-4b52-b5f2-23e7694953c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6818', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6819', 'nonce': 787648514}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6820', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6821', 'nonce': 787648514}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6824', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6825', 'nonce': 787648514}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6822', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6823', 'nonce': 787648514}]}, 'public_addr': '172.21.15.175:6819/787648514', 'cluster_addr': '172.21.15.175:6821/787648514', 'heartbeat_back_addr': '172.21.15.175:6825/787648514', 'heartbeat_front_addr': '172.21.15.175:6823/787648514', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '26d2142e-419c-4da3-9c28-ac442b1b66fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6809', 'nonce': 1424462696}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6811', 'nonce': 1424462696}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6815', 'nonce': 1424462696}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6813', 'nonce': 1424462696}]}, 'public_addr': '172.21.15.175:6809/1424462696', 'cluster_addr': '172.21.15.175:6811/1424462696', 'heartbeat_back_addr': '172.21.15.175:6815/1424462696', 'heartbeat_front_addr': '172.21.15.175:6813/1424462696', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:36.004700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.958497+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.988302+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.993971+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.945648+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:36.366302+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.852613+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.713697+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:09:54.171 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T23:09:54.182 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:54.182+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:09:54.247 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:09:54.282 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:54.282+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:54.283 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:54.283 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:54.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:09:54.383 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:54.382+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:09:54.455 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:54.455 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:54.468 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:09:54.477 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:09:54.477 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:09:54.484 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:54.482+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:09:54.503 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:09:54.503 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:09:54.529 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:09:54.529 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:09:54.584 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:54.582+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:54.586 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:54.587 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:54.599 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:09:54.691 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:54.690+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:09:54.759 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:54.759 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:54.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:09:54.786 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:54.786+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:54.804 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:09:54.887 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:54.886+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:54.912 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:54.912 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:54.924 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:09:54.987 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:54.986+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:09:55.040 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:55.040 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:55.053 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:09:55.099 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:09:55.098+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:09:55.167 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:55.167 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:55.179 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:09:55.223 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:55.222+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:09:55.290 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:55.286+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:55.346 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:55.346 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:55.365 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:09:55.391 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:55.390+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:55.493 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:55.494+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:55.523 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:55.523 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:55.538 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:09:55.593 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:55.594+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:55.661 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:55.661 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:55.675 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:09:55.694 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:55.690+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:09:55.795 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:55.794+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:09:55.809 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:55.809 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:55.823 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:09:55.896 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:55.894+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:09:55.945 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:55.945 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:55.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:09:56.009 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:56.010+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:09:56.082 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:56.082 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:56.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:09:56.097 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:56.094+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:09:56.198 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:56.198+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:56.257 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:56.257 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:56.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:09:56.313 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:56.314+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:56.401 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:56.398+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:56.432 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:56.432 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:56.446 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:09:56.537 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:56.538+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:56.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:09:56.602 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:56.602+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:09:56.619 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:56.619 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:56.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:09:56.707 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:09:56.708 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:09:56.721 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:56.722+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:09:56.724 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:09:56.724 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:09:56.741 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:09:56.742 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:09:56.761 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:09:56.762 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:09:56.782 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:09:56.782 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:09:56.799 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:56.799 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:56.802 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:09:56.802 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:09:56.803 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:56.802+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:56.813 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:09:56.821 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:09:56.821 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:09:56.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:09:56.839 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:09:56.839 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:09:56.905 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:56.906+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:56.984 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:09:56.984 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:09:56.984 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:09:56.984 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:09:56.984 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:09:56.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:09:57.005 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:57.002+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:57.106 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:57.106+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:09:57.163 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:09:57.164 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:09:57.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:09:57.164 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [] 2024-09-21T23:09:57.164 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:09:57.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:09:57.207 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:57.202+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:57.308 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:57.306+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:57.342 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:09:57.342 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:09:57.342 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:09:57.343 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:09:57.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:09:57.362 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:09:57.435 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:57.434+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:09:57.509 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:57.506+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:09:57.524 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:09:57.524 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:09:57.524 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:09:57.524 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:09:57.524 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:09:57.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:09:57.609 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:57.606+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:57.663 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:41.834073+0000", 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.830113591, 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012160298999999999, 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.664 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 88 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834073+0000", 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834073+0000", 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834074+0000", 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.083e-06 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834080+0000", 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834090+0000", 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0307e-05 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834098+0000", 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1529999999999997e-06 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.843284+0000", 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.009186105 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.843556+0000", 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000272148 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.843970+0000", 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00041329700000000002 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.845894+0000", 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019239249999999999 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.846057+0000", 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016394599999999999 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.846132+0000", 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4072000000000005e-05 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.846233+0000", 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101623 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008107+0000", 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.656079155, 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015013937999999999, 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 109 2024-09-21T23:09:57.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008107+0000", 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008107+0000", 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008109+0000", 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008264+0000", 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000154995 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008327+0000", 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3565999999999994e-05 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008345+0000", 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.015463+0000", 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0071178020000000003 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.015603+0000", 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013992200000000001 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.015809+0000", 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020554199999999999 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021621+0000", 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058124049999999997 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.022854+0000", 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012332739999999999 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023009+0000", 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015451500000000001 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023121+0000", 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011234300000000001 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008376+0000", 2024-09-21T23:09:57.675 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.655810683, 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015067053, 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 110 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008376+0000", 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008376+0000", 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008377+0000", 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.553e-06 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008389+0000", 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1486e-05 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008412+0000", 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3383e-05 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008427+0000", 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5147000000000001e-05 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.017805+0000", 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093776740000000008 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.017939+0000", 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013393300000000001 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.018141+0000", 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000202382 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021637+0000", 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034957289999999999 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023251+0000", 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001613504 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023332+0000", 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0949e-05 2024-09-21T23:09:57.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023443+0000", 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111313 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008486+0000", 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.655700123999999, 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015249881999999999, 2024-09-21T23:09:57.681 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 111 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008486+0000", 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008486+0000", 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008489+0000", 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2129999999999998e-06 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008500+0000", 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1443e-05 2024-09-21T23:09:57.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008523+0000", 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3223000000000001e-05 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008553+0000", 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9726e-05 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.018797+0000", 2024-09-21T23:09:57.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010244385 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.018930+0000", 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000132669 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.019130+0000", 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020025899999999999 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023154+0000", 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0040233220000000002 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023587+0000", 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043302599999999998 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023673+0000", 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6130000000000004e-05 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023736+0000", 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3485999999999995e-05 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008585+0000", 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.655601629, 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015465604000000001, 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.687 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 112 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008585+0000", 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008585+0000", 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008586+0000", 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.274e-06 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008761+0000", 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017528199999999999 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008786+0000", 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4306000000000001e-05 2024-09-21T23:09:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008871+0000", 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4919999999999993e-05 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.019893+0000", 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011022185 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.020131+0000", 2024-09-21T23:09:57.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023844400000000001 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.020521+0000", 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038991699999999999 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023754+0000", 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003233072 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023833+0000", 2024-09-21T23:09:57.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9221999999999995e-05 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023941+0000", 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107633 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024050+0000", 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109349 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008914+0000", 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.655272754, 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015372158, 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 113 2024-09-21T23:09:57.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008914+0000", 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008914+0000", 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008915+0000", 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.516e-06 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008927+0000", 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2113000000000001e-05 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008949+0000", 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1353e-05 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008964+0000", 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.577e-05 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021757+0000", 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012792112 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021897+0000", 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014063899999999999 2024-09-21T23:09:57.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.022102+0000", 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020499799999999999 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024150+0000", 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020473779999999999 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024169+0000", 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8966000000000001e-05 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024234+0000", 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4943000000000005e-05 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024286+0000", 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2370000000000002e-05 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008991+0000", 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.655195108000001, 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019578175, 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-21T23:09:57.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008991+0000", 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008991+0000", 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008992+0000", 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009004+0000", 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1853e-05 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009024+0000", 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9893000000000001e-05 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009038+0000", 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4163e-05 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024360+0000", 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015321942 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024511+0000", 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015044200000000001 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024703+0000", 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019235899999999999 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028089+0000", 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0033858439999999998 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028288+0000", 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000199075 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028429+0000", 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140975 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028570+0000", 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014050900000000001 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.704 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.009063+0000", 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.655123732, 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020902028999999999, 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 115 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009063+0000", 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009063+0000", 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009064+0000", 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.176e-06 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009305+0000", 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024120200000000001 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009327+0000", 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2175999999999999e-05 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009344+0000", 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6376999999999999e-05 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.025359+0000", 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016015775999999999 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.025596+0000", 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000236135 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.025972+0000", 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037593400000000002 2024-09-21T23:09:57.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028246+0000", 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022745899999999999 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029655+0000", 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001408999 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029796+0000", 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014105500000000001 2024-09-21T23:09:57.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029965+0000", 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016860900000000001 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.009374+0000", 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.654812120999999, 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020861524999999999, 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.710 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 116 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009374+0000", 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009374+0000", 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009376+0000", 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.463e-06 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009387+0000", 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1463e-05 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009406+0000", 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8977000000000001e-05 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009420+0000", 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3967e-05 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.027097+0000", 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017676586000000001 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.027271+0000", 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000174659 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.027469+0000", 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000197825 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029614+0000", 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00214496 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029995+0000", 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038081399999999999 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030132+0000", 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013733199999999999 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030236+0000", 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103479 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.715 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.009444+0000", 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.654742932, 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021485691000000001, 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 117 2024-09-21T23:09:57.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009444+0000", 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009444+0000", 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009445+0000", 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.08e-06 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009458+0000", 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.358e-05 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009478+0000", 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9323000000000001e-05 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009497+0000", 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9239999999999999e-05 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028716+0000", 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019218981999999999 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028904+0000", 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018812200000000001 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029109+0000", 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020463799999999999 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030248+0000", 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011393250000000001 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030791+0000", 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000543429 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030872+0000", 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030929+0000", 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6736e-05 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.721 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.485790+0000", 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.178396111, 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012605139, 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 132 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485790+0000", 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485790+0000", 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485791+0000", 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6700000000000003e-07 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485796+0000", 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485805+0000", 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5359999999999992e-06 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485809+0000", 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.16e-06 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.487798+0000", 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019890960000000001 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.487886+0000", 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8182000000000004e-05 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488023+0000", 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013699799999999999 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490497+0000", 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024741540000000001 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498343+0000", 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0078459570000000006 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498372+0000", 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8892000000000001e-05 2024-09-21T23:09:57.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498395+0000", 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3447000000000001e-05 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.178214860000001, 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01262986, 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.727 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 133 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6199999999999996e-07 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485977+0000", 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3880000000000002e-06 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485988+0000", 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1233000000000001e-05 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485995+0000", 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0129999999999996e-06 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488384+0000", 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023886979999999999 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488454+0000", 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0320000000000004e-05 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488569+0000", 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114868 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491869+0000", 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003300028 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498461+0000", 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065922000000000003 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498544+0000", 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3089000000000004e-05 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498601+0000", 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7160999999999997e-05 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.732 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.178145905999999, 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012637317, 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 134 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.733 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:09:57.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0400000000000005e-07 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486112+0000", 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486133+0000", 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0954e-05 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486143+0000", 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0190999999999999e-05 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489159+0000", 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030160999999999999 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489237+0000", 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8597999999999995e-05 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489356+0000", 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118837 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491876+0000", 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00251973 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498631+0000", 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0067548520000000004 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498659+0000", 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7835999999999999e-05 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498678+0000", 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9215000000000002e-05 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486191+0000", 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.177995683000001, 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012553811, 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 135 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486191+0000", 2024-09-21T23:09:57.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486191+0000", 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486192+0000", 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486199+0000", 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.674e-06 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486210+0000", 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1942e-05 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486218+0000", 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0539999999999995e-06 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489939+0000", 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037200699999999998 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490026+0000", 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7724000000000002e-05 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490158+0000", 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013132700000000001 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493021+0000", 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002862945 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498702+0000", 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056812520000000004 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498727+0000", 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4973999999999998e-05 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498745+0000", 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7805999999999999e-05 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486266+0000", 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.177920243000001, 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012596937000000001, 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.744 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 136 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486266+0000", 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486266+0000", 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486267+0000", 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0699999999999996e-07 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486274+0000", 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6109999999999999e-06 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486285+0000", 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0896999999999999e-05 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486294+0000", 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0019999999999995e-06 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490552+0000", 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042588160000000003 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490623+0000", 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0487000000000005e-05 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490727+0000", 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103885 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493029+0000", 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023023750000000002 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498784+0000", 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005754459 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498830+0000", 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6288999999999997e-05 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498863+0000", 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3309000000000002e-05 2024-09-21T23:09:57.749 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486310+0000", 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.177876542, 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01263121, 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.750 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 137 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486310+0000", 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486310+0000", 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486311+0000", 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486388+0000", 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7304000000000004e-05 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486398+0000", 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6770000000000007e-06 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486403+0000", 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.553e-06 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491135+0000", 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047316900000000002 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491221+0000", 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5629000000000003e-05 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491353+0000", 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013197 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494512+0000", 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0031596480000000001 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498896+0000", 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043832419999999999 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498920+0000", 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4601999999999999e-05 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498941+0000", 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1030999999999999e-05 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:09:57.755 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.177759686, 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012620973000000001, 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 138 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486432+0000", 2024-09-21T23:09:57.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3209999999999998e-06 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486441+0000", 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7700000000000007e-06 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486445+0000", 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2699999999999998e-06 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491921+0000", 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054762919999999998 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491990+0000", 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9093999999999999e-05 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492091+0000", 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101072 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494518+0000", 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002426344 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498979+0000", 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0044613919999999998 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499021+0000", 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1644999999999999e-05 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499048+0000", 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7146000000000001e-05 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.177716858, 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012646737, 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.761 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 139 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486474+0000", 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8959999999999996e-06 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486481+0000", 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6850000000000004e-06 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486486+0000", 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1220000000000003e-06 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492487+0000", 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060009369999999996 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492577+0000", 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0098000000000004e-05 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492708+0000", 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013093100000000001 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494526+0000", 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018177829999999999 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499070+0000", 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045442540000000002 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499096+0000", 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6432e-05 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499116+0000", 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0058999999999999e-05 2024-09-21T23:09:57.766 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.177692852, 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012957738, 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.767 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 140 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7e-07 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486537+0000", 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3307999999999999e-05 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486544+0000", 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8040000000000004e-06 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486548+0000", 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3669999999999999e-06 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493081+0000", 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065326919999999997 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493151+0000", 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9716999999999997e-05 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493253+0000", 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101832 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494530+0000", 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00127761 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499404+0000", 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004873835 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499430+0000", 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5533000000000001e-05 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499451+0000", 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1670000000000001e-05 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:09:57.772 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486558+0000", 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.177628164, 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013644119, 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 141 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486558+0000", 2024-09-21T23:09:57.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486558+0000", 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486559+0000", 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.75e-07 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486562+0000", 2024-09-21T23:09:57.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486575+0000", 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3122000000000001e-05 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486587+0000", 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2087e-05 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493990+0000", 2024-09-21T23:09:57.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0074032000000000004 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494092+0000", 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010121 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494222+0000", 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129825 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:09:57.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494849+0000", 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062788000000000004 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.500155+0000", 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053054529999999999 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.500179+0000", 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4300000000000001e-05 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.500202+0000", 2024-09-21T23:09:57.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3258999999999999e-05 2024-09-21T23:09:57.778 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.778 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.778 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.778 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:09:57.778 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:09:57.778 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:09:57.779 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:09:57.780 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:57.778+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:57.880 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:57.882+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:57.895 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:57.895 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:09:57.895 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:09:57.895 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:57.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:09:57.981 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:57.982+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:58.024 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:58.024 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:09:58.024 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:09:58.025 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:09:58.025 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:58.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:09:58.083 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:58.082+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:58.158 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:58.158 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:09:58.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:09:58.158 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.697884+0000", 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.459004917000001, 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041145219999999998, 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697884+0000", 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697884+0000", 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697886+0000", 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6139999999999999e-06 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697921+0000", 2024-09-21T23:09:58.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5318999999999998e-05 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697951+0000", 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9366999999999999e-05 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697974+0000", 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3093000000000001e-05 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.698116+0000", 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014264900000000001 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.698152+0000", 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5583000000000001e-05 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.701914+0000", 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037623940000000001 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.701999+0000", 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4503000000000001e-05 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:56 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'34, mlcod=15'29)", 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.764437+0000", 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.392451869999999, 2024-09-21T23:09:58.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044487579999999997, 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764437+0000", 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764437+0000", 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764439+0000", 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.223e-06 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764448+0000", 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8200000000000003e-06 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764467+0000", 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9709999999999999e-05 2024-09-21T23:09:58.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764478+0000", 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0380000000000001e-05 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764569+0000", 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1693000000000005e-05 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764609+0000", 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9619000000000001e-05 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.768804+0000", 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041955639999999997 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.768886+0000", 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1749000000000006e-05 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.362051484999999, 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055137770000000001, 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794840+0000", 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.773e-06 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794892+0000", 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2266999999999999e-05 2024-09-21T23:09:58.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794921+0000", 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9493e-05 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794935+0000", 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.341e-05 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.795055+0000", 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120192 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.795106+0000", 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0875999999999999e-05 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.800255+0000", 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051487499999999997 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.800352+0000", 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7015999999999994e-05 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:71 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'45, mlcod=15'43)", 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.804538+0000", 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.352351783, 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042590070000000004, 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.172 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804538+0000", 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804538+0000", 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804539+0000", 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7430000000000001e-06 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804593+0000", 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3720000000000001e-05 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804622+0000", 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9252999999999999e-05 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804636+0000", 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3716999999999999e-05 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804757+0000", 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012062599999999999 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804808+0000", 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1814999999999998e-05 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808698+0000", 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0038892570000000001 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808797+0000", 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8876000000000007e-05 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.176 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:73 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'47, mlcod=15'43)", 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.807632+0000", 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.349257550000001, 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004089695, 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807632+0000", 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807632+0000", 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807634+0000", 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7260000000000001e-06 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807686+0000", 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2099999999999999e-05 2024-09-21T23:09:58.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807715+0000", 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9417e-05 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807729+0000", 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3692999999999999e-05 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807854+0000", 2024-09-21T23:09:58.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012536599999999999 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807905+0000", 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0849000000000001e-05 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.811629+0000", 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037243770000000001 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.811721+0000", 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2167e-05 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:74 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'48, mlcod=15'43)", 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.347967390000001, 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005385875, 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.181 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808923+0000", 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3260000000000001e-06 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808945+0000", 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1557e-05 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808967+0000", 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1892999999999999e-05 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808978+0000", 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-05 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.809110+0000", 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013212200000000001 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.809176+0000", 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6600000000000006e-05 2024-09-21T23:09:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.814232+0000", 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050558340000000004 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.814308+0000", 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5673000000000005e-05 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:109 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'71, mlcod=15'67)", 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.014090+0000", 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.142799386, 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045428869999999998, 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014090+0000", 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014090+0000", 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014091+0000", 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6800000000000009e-07 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014155+0000", 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3707000000000005e-05 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014170+0000", 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5756e-05 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014181+0000", 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0986e-05 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014272+0000", 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0660000000000003e-05 2024-09-21T23:09:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014295+0000", 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3193e-05 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.018522+0000", 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042264149999999999 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.018633+0000", 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111202 2024-09-21T23:09:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:130 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'88, mlcod=15'85)", 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.671091334, 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070102289999999998, 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485799+0000", 2024-09-21T23:09:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.347e-06 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485864+0000", 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799000000000001e-05 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485882+0000", 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485898+0000", 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6039999999999999e-05 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486068+0000", 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000169815 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486119+0000", 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0599000000000001e-05 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492744+0000", 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066256759999999996 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492808+0000", 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3765999999999999e-05 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:131 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'89, mlcod=15'85)", 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.670954918, 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069196989999999996, 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485935+0000", 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486164+0000", 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000228571 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486204+0000", 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9802999999999998e-05 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486220+0000", 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6820000000000002e-05 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486789+0000", 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00056830900000000002 2024-09-21T23:09:58.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486835+0000", 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6643e-05 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492814+0000", 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059788030000000004 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492854+0000", 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.994e-05 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:132 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'90, mlcod=15'85)", 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.670619324, 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0099194130000000002, 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.199 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486271+0000", 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-21T23:09:58.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486281+0000", 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0207000000000001e-05 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486300+0000", 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9035999999999999e-05 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486313+0000", 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2986999999999999e-05 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487723+0000", 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001409532 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487774+0000", 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0966e-05 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496125+0000", 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083516130000000008 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496189+0000", 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4048999999999996e-05 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.203 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:133 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'91, mlcod=15'85)", 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.670123735000001, 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094674270000000005, 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486767+0000", 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.35e-06 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486779+0000", 2024-09-21T23:09:58.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1847e-05 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486800+0000", 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1413e-05 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486815+0000", 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457000000000001e-05 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488394+0000", 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001579247 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488444+0000", 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0206e-05 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496195+0000", 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077509340000000001 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496233+0000", 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7972999999999997e-05 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.669281810999999, 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087130990000000002, 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487609+0000", 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.367e-06 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487679+0000", 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9702000000000005e-05 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487702+0000", 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3626999999999999e-05 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487722+0000", 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9366000000000001e-05 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488981+0000", 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012596300000000001 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489033+0000", 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1910000000000003e-05 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496237+0000", 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072040580000000002 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496321+0000", 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3438999999999999e-05 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:135 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'93, mlcod=15'86)", 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.668522339999999, 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080451580000000002, 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488368+0000", 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3e-06 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488380+0000", 2024-09-21T23:09:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1976e-05 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488402+0000", 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1997e-05 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488417+0000", 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4986e-05 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489990+0000", 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015723079999999999 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490021+0000", 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1479999999999997e-05 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496328+0000", 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063071409999999996 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496412+0000", 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999997e-05 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'86)", 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.667956082, 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075262560000000003, 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.217 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488935+0000", 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2640000000000001e-06 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488944+0000", 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488961+0000", 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7159999999999998e-05 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488972+0000", 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.042e-05 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490619+0000", 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00164788 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490651+0000", 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1869999999999998e-05 2024-09-21T23:09:58.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496420+0000", 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057688230000000002 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496460+0000", 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9412999999999997e-05 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'86)", 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.667232650000001, 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068427269999999998, 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489658+0000", 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.703e-06 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489712+0000", 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4011999999999997e-05 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489741+0000", 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8144e-05 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489755+0000", 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4270000000000001e-05 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491279+0000", 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001523834 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491307+0000", 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8180000000000001e-05 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496464+0000", 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051571480000000003 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496499+0000", 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5435999999999998e-05 2024-09-21T23:09:58.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:138 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'96, mlcod=15'86)", 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.666590041999999, 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062358350000000003, 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490301+0000", 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.917e-06 2024-09-21T23:09:58.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490315+0000", 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490344+0000", 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490361+0000", 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6883000000000001e-05 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492263+0000", 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001902405 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492296+0000", 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2650000000000001e-05 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496504+0000", 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042075949999999997 2024-09-21T23:09:58.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496535+0000", 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1506e-05 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:139 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'97, mlcod=15'86)", 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.665973177, 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056577900000000002, 2024-09-21T23:09:58.230 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490918+0000", 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3969999999999999e-06 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490929+0000", 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1443e-05 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490953+0000", 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4023e-05 2024-09-21T23:09:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490970+0000", 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6609999999999999e-05 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492980+0000", 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020102779999999999 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493012+0000", 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1846e-05 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496539+0000", 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035270430000000001 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496574+0000", 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5150000000000001e-05 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:140 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'98, mlcod=15'86)", 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.665352181999999, 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057984830000000001, 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491539+0000", 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3629999999999999e-06 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491661+0000", 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012253899999999999 2024-09-21T23:09:58.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491684+0000", 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3173999999999999e-05 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491701+0000", 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7105999999999999e-05 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493648+0000", 2024-09-21T23:09:58.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001946309 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493677+0000", 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9586000000000001e-05 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.497248+0000", 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035708100000000002 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.497336+0000", 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7596000000000006e-05 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:141 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'99, mlcod=15'86)", 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.664470299, 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005692557, 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492420+0000", 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492429+0000", 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0629999999999995e-06 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492446+0000", 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6307000000000002e-05 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492457+0000", 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.154e-05 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496632+0000", 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041749910000000003 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496661+0000", 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9084000000000001e-05 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.498022+0000", 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013602589999999999 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.498112+0000", 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0053e-05 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T23:09:58.243 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.77704995399999999, 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063068619999999999, 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379842+0000", 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1780000000000002e-06 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379856+0000", 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4936999999999999e-05 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379878+0000", 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1248999999999999e-05 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379888+0000", 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0302999999999999e-05 2024-09-21T23:09:58.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379983+0000", 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4987000000000004e-05 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379992+0000", 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.568e-06 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383095+0000", 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031032379999999999 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383305+0000", 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020978199999999999 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383621+0000", 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000316562 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.385245+0000", 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016241459999999999 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.385925+0000", 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067988399999999996 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.386038+0000", 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113081 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.386146+0000", 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107947 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.250 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:58.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:09:58.252 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:58.250+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:58.352 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:58.350+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:58.415 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:09:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:09:58.415 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:58.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:09:58.471 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:58.470+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:09:58.553 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:58.554+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:58.596 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:58.596 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:09:58.596 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:09:58.596 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:09:58.597 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:58.609 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:09:58.654 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:58.654+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:46.720844+0000", 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.007102219, 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022363890000000001, 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720844+0000", 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720844+0000", 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720845+0000", 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720853+0000", 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4670000000000004e-06 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720884+0000", 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0552999999999997e-05 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720903+0000", 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8998999999999999e-05 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.721051+0000", 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148176 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.721103+0000", 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2333000000000002e-05 2024-09-21T23:09:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.722979+0000", 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001876445 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.723080+0000", 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010086600000000001 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.006616489000001, 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002253339, 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.75e-07 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721347+0000", 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6390999999999999e-05 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721375+0000", 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8263e-05 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721398+0000", 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2631000000000001e-05 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721547+0000", 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014990700000000001 2024-09-21T23:09:58.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721608+0000", 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0646999999999997e-05 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.723466+0000", 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018583759999999999 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.723583+0000", 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116549 2024-09-21T23:09:58.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.356371472, 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00125829, 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:09:58.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3300000000000003e-07 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371585+0000", 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0299000000000001e-05 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371595+0000", 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6569999999999992e-06 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371607+0000", 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1664999999999999e-05 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371698+0000", 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1651999999999996e-05 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371724+0000", 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5508999999999999e-05 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.372707+0000", 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00098331400000000011 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:09:58.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.372833+0000", 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000125761 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:09:58.740 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:58.741 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:09:58.789 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:09:58.786+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:09:58.855 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:58.854+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:58.857 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:09:58.866 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:09:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:09:58.866 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:09:58.866 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:09:58.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:09:58.967 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:58.966+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:58.992 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:09:58.992 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:09:59.008 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:09:59.008 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:09:59.024 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:09:59.025 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:09:59.036 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:59.036 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:59.041 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:09:59.041 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:09:59.049 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:09:59.056 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:59.054+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:59.061 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:09:59.061 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:09:59.082 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:09:59.082 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:09:59.103 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:09:59.103 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:09:59.123 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:09:59.124 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:09:59.171 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T23:09:59.172 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:09:59.176 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:59.176 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:59.176 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:59.174+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:59.192 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:09:59.257 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:09:59.258+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:09:59.293 INFO:teuthology.orchestra.run.smithi150.stdout:1 write 0x207205a thru 0x207ba36 (0x99dd bytes) 2024-09-21T23:09:59.354 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:59.354 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:59.358 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:59.354+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:09:59.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:09:59.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:09:59.459 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:09:59.458+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:09:59.501 INFO:teuthology.orchestra.run.smithi150.stdout:2 punch from 0x311f46 to 0x312165, (0x21f bytes) 2024-09-21T23:09:59.502 INFO:teuthology.orchestra.run.smithi150.stdout:3 read 0x51c2b5 thru 0x527d26 (0xba72 bytes) 2024-09-21T23:09:59.502 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0x2865c6a 2024-09-21T23:09:59.502 INFO:teuthology.orchestra.run.smithi150.stdout:4 trunc from 0x207ba37 to 0x2865c6a 2024-09-21T23:09:59.515 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:09:59.516 INFO:teuthology.orchestra.run.smithi150.stdout:7 punch from 0x1d2ccf4 to 0x1d38660, (0xb96c bytes) 2024-09-21T23:09:59.519 INFO:teuthology.orchestra.run.smithi150.stdout:9 trunc from 0x2865c6a to 0x9418f7 2024-09-21T23:09:59.533 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:59.533 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:59.539 INFO:teuthology.orchestra.run.smithi150.stdout:11 punch from 0x2fa8e8 to 0x2fb2fd, (0xa15 bytes) 2024-09-21T23:09:59.539 INFO:teuthology.orchestra.run.smithi150.stdout:13 read 0x7beadf thru 0x7c0ce1 (0x2203 bytes) 2024-09-21T23:09:59.551 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:09:59.559 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:09:59.558+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:09:59.565 INFO:teuthology.orchestra.run.smithi150.stdout:14 write 0xac43912 thru 0xac4d339 (0x9a28 bytes) 2024-09-21T23:09:59.571 INFO:teuthology.orchestra.run.smithi150.stdout:15 read 0x19c8e85 thru 0x19d04f7 (0x7673 bytes) 2024-09-21T23:09:59.571 INFO:teuthology.orchestra.run.smithi150.stdout:16 punch from 0xa2802d5 to 0xa28d27a, (0xcfa5 bytes) 2024-09-21T23:09:59.571 INFO:teuthology.orchestra.run.smithi150.stdout:17 read 0x3196b68 thru 0x31a1413 (0xa8ac bytes) 2024-09-21T23:09:59.571 INFO:teuthology.orchestra.run.smithi150.stdout:18 punch from 0x3725fd8 to 0x3727866, (0x188e bytes) 2024-09-21T23:09:59.571 INFO:teuthology.orchestra.run.smithi150.stdout:19 writesame 0x5d183d4 thru 0x5d25484 data_size 0x5(0xd0b1 bytes) 2024-09-21T23:09:59.579 INFO:teuthology.orchestra.run.smithi150.stdout:21 clone 1 order 25 su 8388608 sc 8 2024-09-21T23:09:59.660 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:09:59.658+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:09:59.663 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:59.664 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:59.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:09:59.761 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:09:59.758+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:09:59.807 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:59.808 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:59.822 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:09:59.861 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:09:59.858+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:09:59.944 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:09:59.944 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:09:59.957 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:09:59.962 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:09:59.958+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:00.062 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:00.062+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:00.081 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:00.081 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:00.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:00.163 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:00.162+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:00.262 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:00.262 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:00.263 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:00.262+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:00.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:00.364 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:00.362+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:00.380 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:10:00.384 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:10:00.408 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:10:00.441 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:00.442 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:00.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:00.505 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:00.506+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:00.566 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:00.562+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:00.581 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:00.582 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:00.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:00.598 INFO:teuthology.orchestra.run.smithi150.stdout:22 write 0x1af8257 thru 0x1b0499a (0xc744 bytes) 2024-09-21T23:10:00.608 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:00.609 INFO:teuthology.orchestra.run.smithi150.stdout:24 read 0x6b0f71a thru 0x6b17cf2 (0x85d9 bytes) 2024-09-21T23:10:00.609 INFO:teuthology.orchestra.run.smithi150.stdout:27 read 0x4245933 thru 0x4253e64 (0xe532 bytes) 2024-09-21T23:10:00.609 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xb3bf218 2024-09-21T23:10:00.609 INFO:teuthology.orchestra.run.smithi150.stdout:28 trunc from 0xac4d33a to 0xb3bf218 2024-09-21T23:10:00.626 INFO:teuthology.orchestra.run.smithi150.stdout:29 write 0xc5f01df thru 0xc5f6116 (0x5f38 bytes) 2024-09-21T23:10:00.633 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:00.633 INFO:teuthology.orchestra.run.smithi150.stdout:32 read 0xbc94c71 thru 0xbca369f (0xea2f bytes) 2024-09-21T23:10:00.633 INFO:teuthology.orchestra.run.smithi150.stdout:33 read 0x1a69e2c thru 0x1a729df (0x8bb4 bytes) 2024-09-21T23:10:00.634 INFO:teuthology.orchestra.run.smithi150.stdout:34 read 0x7425d9b thru 0x7434c51 (0xeeb7 bytes) 2024-09-21T23:10:00.634 INFO:teuthology.orchestra.run.smithi150.stdout:36 write 0x9da7848 thru 0x9db4a0c (0xd1c5 bytes) 2024-09-21T23:10:00.638 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:00.638 INFO:teuthology.orchestra.run.smithi150.stdout:38 write 0x3a1657d thru 0x3a25e0f (0xf893 bytes) 2024-09-21T23:10:00.658 INFO:teuthology.orchestra.run.smithi150.stdout:39 write 0xdec264d thru 0xdec9eed (0x78a1 bytes) 2024-09-21T23:10:00.661 INFO:teuthology.orchestra.run.smithi150.stdout:40 writesame 0x9f7b756 thru 0x9f7e403 data_size 0x2b(0x2cae bytes) 2024-09-21T23:10:00.663 INFO:teuthology.orchestra.run.smithi150.stdout:41 write 0x3c5c641 thru 0x3c669ae (0xa36e bytes) 2024-09-21T23:10:00.665 INFO:teuthology.orchestra.run.smithi150.stdout:42 read 0xb5f3279 thru 0xb601728 (0xe4b0 bytes) 2024-09-21T23:10:00.666 INFO:teuthology.orchestra.run.smithi150.stdout:43 write 0xbfcda34 thru 0xbfd6cf1 (0x92be bytes) 2024-09-21T23:10:00.666 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:00.662+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:00.668 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:00.668 INFO:teuthology.orchestra.run.smithi150.stdout:46 write 0x7dc521f thru 0x7dc936b (0x414d bytes) 2024-09-21T23:10:00.669 INFO:teuthology.orchestra.run.smithi150.stdout:47 read 0xc7e4dc4 thru 0xc7ebd10 (0x6f4d bytes) 2024-09-21T23:10:00.670 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:00.682 INFO:teuthology.orchestra.run.smithi150.stdout:49 write 0xe720a3a thru 0xe7295a9 (0x8b70 bytes) 2024-09-21T23:10:00.687 INFO:teuthology.orchestra.run.smithi150.stdout:50 punch from 0x3bcdcb0 to 0x3bd4243, (0x6593 bytes) 2024-09-21T23:10:00.687 INFO:teuthology.orchestra.run.smithi150.stdout:51 writesame 0x674c018 thru 0x6759f45 data_size 0x35(0xdf2e bytes) 2024-09-21T23:10:00.689 INFO:teuthology.orchestra.run.smithi150.stdout:52 writesame 0xd2305d thru 0xd29097 data_size 0xd(0x603b bytes) 2024-09-21T23:10:00.696 INFO:teuthology.orchestra.run.smithi150.stdout:53 read 0x604cb54 thru 0x605a2d9 (0xd786 bytes) 2024-09-21T23:10:00.697 INFO:teuthology.orchestra.run.smithi150.stdout:56 writesame 0x3f9f919 thru 0x3fac790 data_size 0x4(0xce78 bytes) 2024-09-21T23:10:00.706 INFO:teuthology.orchestra.run.smithi150.stdout:58 writesame 0x728057b thru 0x72825ba data_size 0x30(0x2040 bytes) 2024-09-21T23:10:00.711 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:00.711 INFO:teuthology.orchestra.run.smithi150.stdout:62 write 0x4a82c11 thru 0x4a89e6a (0x725a bytes) 2024-09-21T23:10:00.714 INFO:teuthology.orchestra.run.smithi150.stdout:63 write 0xac0c653 thru 0xac19a6c (0xd41a bytes) 2024-09-21T23:10:00.721 INFO:teuthology.orchestra.run.smithi150.stdout:64 punch from 0xa187cd3 to 0xa18baa1, (0x3dce bytes) 2024-09-21T23:10:00.721 INFO:teuthology.orchestra.run.smithi150.stdout:66 write 0xc83b653 thru 0xc8441f1 (0x8b9f bytes) 2024-09-21T23:10:00.724 INFO:teuthology.orchestra.run.smithi150.stdout:67 writesame 0x9a49099 thru 0x9a4d41c data_size 0x3a(0x4384 bytes) 2024-09-21T23:10:00.727 INFO:teuthology.orchestra.run.smithi150.stdout:68 punch from 0xa1c87d9 to 0xa1ce3bb, (0x5be2 bytes) 2024-09-21T23:10:00.727 INFO:teuthology.orchestra.run.smithi150.stdout:69 writesame 0x6779d34 thru 0x677bdc6 data_size 0x1(0x2093 bytes) 2024-09-21T23:10:00.737 INFO:teuthology.orchestra.run.smithi150.stdout:70 punch from 0x57bd8b8 to 0x57ca331, (0xca79 bytes) 2024-09-21T23:10:00.737 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:00.737 INFO:teuthology.orchestra.run.smithi150.stdout:74 read 0x4129cf0 thru 0x41395b8 (0xf8c9 bytes) 2024-09-21T23:10:00.738 INFO:teuthology.orchestra.run.smithi150.stdout:75 read 0x910b629 thru 0x910f8da (0x42b2 bytes) 2024-09-21T23:10:00.738 INFO:teuthology.orchestra.run.smithi150.stdout:76 punch from 0x23bf254 to 0x23cbdd3, (0xcb7f bytes) 2024-09-21T23:10:00.738 INFO:teuthology.orchestra.run.smithi150.stdout:77 read 0x5ca9f2e thru 0x5cb57e2 (0xb8b5 bytes) 2024-09-21T23:10:00.751 INFO:teuthology.orchestra.run.smithi150.stdout:79 trunc from 0xe7295aa to 0x6f80449 2024-09-21T23:10:00.767 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:00.766+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:00.776 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:00.776 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:00.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:00.791 INFO:teuthology.orchestra.run.smithi150.stdout:80 writesame 0xb08a829 thru 0xb093133 data_size 0x1(0x890b bytes) 2024-09-21T23:10:00.806 INFO:teuthology.orchestra.run.smithi150.stdout:81 write 0xb8b227a thru 0xb8b2cd7 (0xa5e bytes) 2024-09-21T23:10:00.808 INFO:teuthology.orchestra.run.smithi150.stdout:82 punch from 0x4d1dcfe to 0x4d28b7c, (0xae7e bytes) 2024-09-21T23:10:00.812 INFO:teuthology.orchestra.run.smithi150.stdout:83 trunc from 0xb8b2cd8 to 0x9baf7a7 2024-09-21T23:10:00.826 INFO:teuthology.orchestra.run.smithi150.stdout:84 read 0x592019d thru 0x592ddca (0xdc2e bytes) 2024-09-21T23:10:00.827 INFO:teuthology.orchestra.run.smithi150.stdout:85 write 0x68619e8 thru 0x6867466 (0x5a7f bytes) 2024-09-21T23:10:00.830 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xc84f74a 2024-09-21T23:10:00.836 INFO:teuthology.orchestra.run.smithi150.stdout:87 trunc from 0x9baf7a7 to 0xc84f74a 2024-09-21T23:10:00.847 INFO:teuthology.orchestra.run.smithi150.stdout:88 read 0xd5a23d thru 0xd5bec5 (0x1c89 bytes) 2024-09-21T23:10:00.848 INFO:teuthology.orchestra.run.smithi150.stdout:89 writesame 0xc45b7e3 thru 0xc467627 data_size 0x1(0xbe45 bytes) 2024-09-21T23:10:00.868 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:00.866+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:00.908 INFO:teuthology.orchestra.run.smithi150.stdout:90 writesame 0x9357507 thru 0x935c312 data_size 0x1e(0x4e0c bytes) 2024-09-21T23:10:00.912 INFO:teuthology.orchestra.run.smithi150.stdout:92 write 0x31c9b46 thru 0x31d5c4f (0xc10a bytes) 2024-09-21T23:10:00.916 INFO:teuthology.orchestra.run.smithi150.stdout:93 read 0xc43b018 thru 0xc43b460 (0x449 bytes) 2024-09-21T23:10:00.924 INFO:teuthology.orchestra.run.smithi150.stdout:94 trunc from 0xc84f74a to 0xa022b3a 2024-09-21T23:10:00.961 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:00.961 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:00.967 INFO:teuthology.orchestra.run.smithi150.stdout:95 trunc from 0xa022b3a to 0xa6e38c 2024-09-21T23:10:00.969 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:00.966+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:00.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:01.005 INFO:teuthology.orchestra.run.smithi150.stdout:97 punch from 0x3a0095 to 0x3a5b3c, (0x5aa7 bytes) 2024-09-21T23:10:01.021 INFO:teuthology.orchestra.run.smithi150.stdout:98 write 0x2d08d76 thru 0x2d14523 (0xb7ae bytes) 2024-09-21T23:10:01.031 INFO:teuthology.orchestra.run.smithi150.stdout:100 punch from 0x181fc98 to 0x182caa1, (0xce09 bytes) 2024-09-21T23:10:01.062 INFO:teuthology.orchestra.run.smithi150.stdout:103 writesame 0x8be0826 thru 0x8bec6e0 data_size 0x1(0xbebb bytes) 2024-09-21T23:10:01.065 INFO:teuthology.orchestra.run.smithi150.stdout:105 punch from 0x16a593c to 0x16adc81, (0x8345 bytes) 2024-09-21T23:10:01.069 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:01.066+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:01.082 INFO:teuthology.orchestra.run.smithi150.stdout:106 writesame 0x916fd44 thru 0x917a105 data_size 0x66(0xa3c2 bytes) 2024-09-21T23:10:01.082 INFO:teuthology.orchestra.run.smithi150.stdout:107 punch from 0x85adf8e to 0x85bcdfa, (0xee6c bytes) 2024-09-21T23:10:01.082 INFO:teuthology.orchestra.run.smithi150.stdout:108 punch from 0x24d2c81 to 0x24d505a, (0x23d9 bytes) 2024-09-21T23:10:01.083 INFO:teuthology.orchestra.run.smithi150.stdout:109 writesame 0x7046ef6 thru 0x704f013 data_size 0x15(0x811e bytes) 2024-09-21T23:10:01.093 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.107 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:01.108 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:01.115 INFO:teuthology.orchestra.run.smithi150.stdout:112 writesame 0xafbf8c6 thru 0xafc001f data_size 0x1(0x75a bytes) 2024-09-21T23:10:01.118 INFO:teuthology.orchestra.run.smithi150.stdout:113 punch from 0x280f78e to 0x28117f7, (0x2069 bytes) 2024-09-21T23:10:01.118 INFO:teuthology.orchestra.run.smithi150.stdout:114 read 0x3760b77 thru 0x3765de8 (0x5272 bytes) 2024-09-21T23:10:01.119 INFO:teuthology.orchestra.run.smithi150.stdout:115 write 0x8fb76f3 thru 0x8fc39fe (0xc30c bytes) 2024-09-21T23:10:01.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:01.123 INFO:teuthology.orchestra.run.smithi150.stdout:117 read 0x8a24974 thru 0x8a30170 (0xb7fd bytes) 2024-09-21T23:10:01.124 INFO:teuthology.orchestra.run.smithi150.stdout:118 read 0x5f0ff23 thru 0x5f1aa9d (0xab7b bytes) 2024-09-21T23:10:01.124 INFO:teuthology.orchestra.run.smithi150.stdout:121 read 0x9c86594 thru 0x9c86b34 (0x5a1 bytes) 2024-09-21T23:10:01.136 INFO:teuthology.orchestra.run.smithi150.stdout:122 write 0xb024fd5 thru 0xb02fb36 (0xab62 bytes) 2024-09-21T23:10:01.139 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.139 INFO:teuthology.orchestra.run.smithi150.stdout:124 punch from 0x96f7032 to 0x96fd718, (0x66e6 bytes) 2024-09-21T23:10:01.145 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:01.150 INFO:teuthology.orchestra.run.smithi150.stdout:125 write 0xb6b3cc8 thru 0xb6c2210 (0xe549 bytes) 2024-09-21T23:10:01.154 INFO:teuthology.orchestra.run.smithi150.stdout:127 read 0xaf20994 thru 0xaf27b20 (0x718d bytes) 2024-09-21T23:10:01.154 INFO:teuthology.orchestra.run.smithi150.stdout:128 read 0x633bef thru 0x640ba2 (0xcfb4 bytes) 2024-09-21T23:10:01.156 INFO:teuthology.orchestra.run.smithi150.stdout:130 read 0x4616705 thru 0x4617bd0 (0x14cc bytes) 2024-09-21T23:10:01.157 INFO:teuthology.orchestra.run.smithi150.stdout:131 read 0x50f4351 thru 0x5103177 (0xee27 bytes) 2024-09-21T23:10:01.157 INFO:teuthology.orchestra.run.smithi150.stdout:133 punch from 0x414b241 to 0x4150f12, (0x5cd1 bytes) 2024-09-21T23:10:01.157 INFO:teuthology.orchestra.run.smithi150.stdout:134 read 0x2d196e2 thru 0x2d27626 (0xdf45 bytes) 2024-09-21T23:10:01.157 INFO:teuthology.orchestra.run.smithi150.stdout:135 write 0x464f465 thru 0x465a9aa (0xb546 bytes) 2024-09-21T23:10:01.170 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:01.170+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:01.173 INFO:teuthology.orchestra.run.smithi150.stdout:136 write 0xcf5121e thru 0xcf56a34 (0x5817 bytes) 2024-09-21T23:10:01.185 INFO:teuthology.orchestra.run.smithi150.stdout:137 write 0xe009e2b thru 0xe00eb75 (0x4d4b bytes) 2024-09-21T23:10:01.190 INFO:teuthology.orchestra.run.smithi150.stdout:138 punch from 0x67d2a83 to 0x67d8bd0, (0x614d bytes) 2024-09-21T23:10:01.190 INFO:teuthology.orchestra.run.smithi150.stdout:140 read 0xdddf68b thru 0xddecb87 (0xd4fd bytes) 2024-09-21T23:10:01.202 INFO:teuthology.orchestra.run.smithi150.stdout:141 write 0xe968e3a thru 0xe96d48e (0x4655 bytes) 2024-09-21T23:10:01.205 INFO:teuthology.orchestra.run.smithi150.stdout:142 writesame 0xbae3717 thru 0xbaf03fc data_size 0x1(0xcce6 bytes) 2024-09-21T23:10:01.245 INFO:teuthology.orchestra.run.smithi150.stdout:143 write 0xed773 thru 0xfa7ca (0xd058 bytes) 2024-09-21T23:10:01.250 INFO:teuthology.orchestra.run.smithi150.stdout:144 read 0x6ee8453 thru 0x6ee9f7f (0x1b2d bytes) 2024-09-21T23:10:01.250 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.250 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.250 INFO:teuthology.orchestra.run.smithi150.stdout:147 read 0x2514806 thru 0x251fd24 (0xb51f bytes) 2024-09-21T23:10:01.250 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:01.250 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:01.251 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xcb2d191 2024-09-21T23:10:01.255 INFO:teuthology.orchestra.run.smithi150.stdout:148 trunc from 0xe96d48f to 0xcb2d191 2024-09-21T23:10:01.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:01.268 INFO:teuthology.orchestra.run.smithi150.stdout:149 punch from 0x12c8af9 to 0x12d1e83, (0x938a bytes) 2024-09-21T23:10:01.268 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.268 INFO:teuthology.orchestra.run.smithi150.stdout:152 read 0x2b4ece3 thru 0x2b583b3 (0x96d1 bytes) 2024-09-21T23:10:01.269 INFO:teuthology.orchestra.run.smithi150.stdout:154 read 0x6945332 thru 0x69457d1 (0x4a0 bytes) 2024-09-21T23:10:01.269 INFO:teuthology.orchestra.run.smithi150.stdout:156 writesame 0x954038b thru 0x9545fff data_size 0x1(0x5c75 bytes) 2024-09-21T23:10:01.271 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:01.270+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:01.287 INFO:teuthology.orchestra.run.smithi150.stdout:158 write 0x97a1ab5 thru 0x97a919b (0x76e7 bytes) 2024-09-21T23:10:01.308 INFO:teuthology.orchestra.run.smithi150.stdout:159 trunc from 0xcb2d191 to 0x17c3981 2024-09-21T23:10:01.311 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:01.311 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:01.329 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:01.329 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:01.337 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.338 INFO:teuthology.orchestra.run.smithi150.stdout:164 read 0x1181eb6 thru 0x118ea39 (0xcb84 bytes) 2024-09-21T23:10:01.338 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.338 INFO:teuthology.orchestra.run.smithi150.stdout:168 punch from 0x938792 to 0x9415d0, (0x8e3e bytes) 2024-09-21T23:10:01.347 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:01.347 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:01.354 INFO:teuthology.orchestra.run.smithi150.stdout:169 write 0x6b38193 thru 0x6b4040b (0x8279 bytes) 2024-09-21T23:10:01.360 INFO:teuthology.orchestra.run.smithi150.stdout:170 read 0x303af1b thru 0x3047a48 (0xcb2e bytes) 2024-09-21T23:10:01.360 INFO:teuthology.orchestra.run.smithi150.stdout:171 punch from 0xf1d302 to 0xf2be73, (0xeb71 bytes) 2024-09-21T23:10:01.360 INFO:teuthology.orchestra.run.smithi150.stdout:172 punch from 0x5e49abe to 0x5e55556, (0xba98 bytes) 2024-09-21T23:10:01.361 INFO:teuthology.orchestra.run.smithi150.stdout:174 write 0x57acef3 thru 0x57b51c3 (0x82d1 bytes) 2024-09-21T23:10:01.363 INFO:teuthology.orchestra.run.smithi150.stdout:175 write 0x8c83d3 thru 0x8d7bf9 (0xf827 bytes) 2024-09-21T23:10:01.365 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:01.365 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:01.367 INFO:teuthology.orchestra.run.smithi150.stdout:176 read 0x190d270 thru 0x191bef7 (0xec88 bytes) 2024-09-21T23:10:01.367 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.367 INFO:teuthology.orchestra.run.smithi150.stdout:179 punch from 0xb8ac34 to 0xb91b61, (0x6f2d bytes) 2024-09-21T23:10:01.367 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.367 INFO:teuthology.orchestra.run.smithi150.stdout:181 writesame 0x1e2b810 thru 0x1e33fba data_size 0x33(0x87ab bytes) 2024-09-21T23:10:01.371 INFO:teuthology.orchestra.run.smithi150.stdout:182 punch from 0x2b22e2b to 0x2b31f91, (0xf166 bytes) 2024-09-21T23:10:01.371 INFO:teuthology.orchestra.run.smithi150.stdout:183 read 0x472c147 thru 0x47317a8 (0x5662 bytes) 2024-09-21T23:10:01.372 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:01.370+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:01.381 INFO:teuthology.orchestra.run.smithi150.stdout:184 writesame 0x7c82a3e thru 0x7c888db data_size 0x21(0x5e9e bytes) 2024-09-21T23:10:01.381 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.386 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:10:01.386 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:10:01.390 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:10:01.391 INFO:teuthology.orchestra.run.smithi150.stdout:186 trunc from 0x7c888dc to 0x107eacb 2024-09-21T23:10:01.411 INFO:teuthology.orchestra.run.smithi150.stdout:187 write 0x7ca1e5 thru 0x7d7d2d (0xdb49 bytes) 2024-09-21T23:10:01.413 INFO:teuthology.orchestra.run.smithi150.stdout:188 read 0x6912dd thru 0x691539 (0x25d bytes) 2024-09-21T23:10:01.414 INFO:teuthology.orchestra.run.smithi150.stdout:189 read 0xfc5212 thru 0xfc5620 (0x40f bytes) 2024-09-21T23:10:01.416 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.417 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:10:01.418 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:01.418 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:01.423 INFO:teuthology.orchestra.run.smithi150.stdout:192 write 0x1be9199 thru 0x1bf7235 (0xe09d bytes) 2024-09-21T23:10:01.428 INFO:teuthology.orchestra.run.smithi150.stdout:194 read 0x2484e2 thru 0x24943c (0xf5b bytes) 2024-09-21T23:10:01.428 INFO:teuthology.orchestra.run.smithi150.stdout:195 read 0x48c08b thru 0x493b69 (0x7adf bytes) 2024-09-21T23:10:01.439 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:01.439 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:01.442 INFO:teuthology.orchestra.run.smithi150.stdout:196 write 0x5f1f69e thru 0x5f2a2a8 (0xac0b bytes) 2024-09-21T23:10:01.443 INFO:teuthology.orchestra.run.smithi150.stdout:197 read 0x41a12be thru 0x41a90f8 (0x7e3b bytes) 2024-09-21T23:10:01.444 INFO:teuthology.orchestra.run.smithi150.stdout:200 write 0x58c6edd thru 0x58cd539 (0x665d bytes) 2024-09-21T23:10:01.445 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:01.445 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:01.446 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.446 INFO:teuthology.orchestra.run.smithi150.stdout:202 read 0x7fba85 thru 0x80379c (0x7d18 bytes) 2024-09-21T23:10:01.446 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.446 INFO:teuthology.orchestra.run.smithi150.stdout:206 read 0x5331289 thru 0x533446e (0x31e6 bytes) 2024-09-21T23:10:01.458 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:01.458 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:01.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:01.462 INFO:teuthology.orchestra.run.smithi150.stdout:208 writesame 0xb43d3f9 thru 0xb4486b8 data_size 0x6e(0xb2c0 bytes) 2024-09-21T23:10:01.462 INFO:teuthology.orchestra.run.smithi150.stdout:209 write 0x3a2e06e thru 0x3a3a5fd (0xc590 bytes) 2024-09-21T23:10:01.467 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.467 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.475 INFO:teuthology.orchestra.run.smithi150.stdout:213 trunc from 0xb4486b9 to 0x5d36d16 2024-09-21T23:10:01.485 INFO:teuthology.orchestra.run.smithi150.stdout:215 write 0xa3ff1d thru 0xa429e5 (0x2ac9 bytes) 2024-09-21T23:10:01.487 INFO:teuthology.orchestra.run.smithi150.stdout:216 read 0x59e41f2 thru 0x59ea46c (0x627b bytes) 2024-09-21T23:10:01.487 INFO:teuthology.orchestra.run.smithi150.stdout:219 read 0x5c40bcc thru 0x5c42ef7 (0x232c bytes) 2024-09-21T23:10:01.488 INFO:teuthology.orchestra.run.smithi150.stdout:221 read 0xcdd1ef thru 0xce8449 (0xb25b bytes) 2024-09-21T23:10:01.488 INFO:teuthology.orchestra.run.smithi150.stdout:223 read 0xf8f60d thru 0xf93319 (0x3d0d bytes) 2024-09-21T23:10:01.488 INFO:teuthology.orchestra.run.smithi150.stdout:224 read 0x3d7fc7a thru 0x3d8b477 (0xb7fe bytes) 2024-09-21T23:10:01.489 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:01.489 INFO:teuthology.orchestra.run.smithi150.stdout:229 punch from 0x4e20228 to 0x4e2e81b, (0xe5f3 bytes) 2024-09-21T23:10:01.489 INFO:teuthology.orchestra.run.smithi150.stdout:230 punch from 0x3f7342f to 0x3f82b57, (0xf728 bytes) 2024-09-21T23:10:01.489 INFO:teuthology.orchestra.run.smithi150.stdout:231 punch from 0x772f4e to 0x778094, (0x5146 bytes) 2024-09-21T23:10:01.501 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:01.502+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:01.509 INFO:teuthology.orchestra.run.smithi150.stdout:232 writesame 0xec03d37 thru 0xec0ea34 data_size 0x42(0xacfe bytes) 2024-09-21T23:10:01.509 INFO:teuthology.orchestra.run.smithi150.stdout:234 read 0x6dc8ff0 thru 0x6dd2f6c (0x9f7d bytes) 2024-09-21T23:10:01.509 INFO:teuthology.orchestra.run.smithi150.stdout:235 read 0x31b425 thru 0x327d93 (0xc96f bytes) 2024-09-21T23:10:01.510 INFO:teuthology.orchestra.run.smithi150.stdout:237 write 0x54bf68e thru 0x54c858f (0x8f02 bytes) 2024-09-21T23:10:01.516 INFO:teuthology.orchestra.run.smithi150.stdout:238 trunc from 0xec0ea35 to 0xbc63fa9 2024-09-21T23:10:01.525 INFO:teuthology.orchestra.run.smithi150.stdout:239 write 0xb5aac6c thru 0xb5ad62e (0x29c3 bytes) 2024-09-21T23:10:01.529 INFO:teuthology.orchestra.run.smithi150.stdout:240 write 0x5ec57fd thru 0x5ecb2ec (0x5af0 bytes) 2024-09-21T23:10:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:244 read 0x2ffb4e8 thru 0x2ffe450 (0x2f69 bytes) 2024-09-21T23:10:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:245 write 0x80fb463 thru 0x8108072 (0xcc10 bytes) 2024-09-21T23:10:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:246 clone 2 order 25 su 1048576 sc 2 2024-09-21T23:10:01.574 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:01.574+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:01.630 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:41.834073+0000", 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.795857759, 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012160298999999999, 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 88 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834073+0000", 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834073+0000", 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834074+0000", 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.083e-06 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834080+0000", 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834090+0000", 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0307e-05 2024-09-21T23:10:01.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.834098+0000", 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1529999999999997e-06 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.843284+0000", 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.009186105 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.843556+0000", 2024-09-21T23:10:01.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000272148 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.843970+0000", 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00041329700000000002 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.845894+0000", 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019239249999999999 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.846057+0000", 2024-09-21T23:10:01.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016394599999999999 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.846132+0000", 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4072000000000005e-05 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:41.846233+0000", 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101623 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008107+0000", 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.621823323000001, 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015013937999999999, 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 109 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008107+0000", 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008107+0000", 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008109+0000", 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008264+0000", 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000154995 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008327+0000", 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3565999999999994e-05 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008345+0000", 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.015463+0000", 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0071178020000000003 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.015603+0000", 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013992200000000001 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.015809+0000", 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020554199999999999 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021621+0000", 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058124049999999997 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.022854+0000", 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012332739999999999 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023009+0000", 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015451500000000001 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023121+0000", 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011234300000000001 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008376+0000", 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.621554850999999, 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015067053, 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.642 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 110 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008376+0000", 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008376+0000", 2024-09-21T23:10:01.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008377+0000", 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.553e-06 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008389+0000", 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1486e-05 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008412+0000", 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3383e-05 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008427+0000", 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5147000000000001e-05 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.017805+0000", 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0093776740000000008 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.017939+0000", 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013393300000000001 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.018141+0000", 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000202382 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021637+0000", 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034957289999999999 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023251+0000", 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001613504 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023332+0000", 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0949e-05 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023443+0000", 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111313 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.647 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008486+0000", 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.621444292, 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015249881999999999, 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 111 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.648 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008486+0000", 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008486+0000", 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008489+0000", 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2129999999999998e-06 2024-09-21T23:10:01.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008500+0000", 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1443e-05 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008523+0000", 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3223000000000001e-05 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008553+0000", 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9726e-05 2024-09-21T23:10:01.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.018797+0000", 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010244385 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.018930+0000", 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000132669 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.019130+0000", 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020025899999999999 2024-09-21T23:10:01.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023154+0000", 2024-09-21T23:10:01.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0040233220000000002 2024-09-21T23:10:01.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023587+0000", 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043302599999999998 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023673+0000", 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6130000000000004e-05 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023736+0000", 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3485999999999995e-05 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.654 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008585+0000", 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.621345797, 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015465604000000001, 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 112 2024-09-21T23:10:01.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008585+0000", 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008585+0000", 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008586+0000", 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.274e-06 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008761+0000", 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017528199999999999 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008786+0000", 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4306000000000001e-05 2024-09-21T23:10:01.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008871+0000", 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4919999999999993e-05 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.019893+0000", 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011022185 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.020131+0000", 2024-09-21T23:10:01.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023844400000000001 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.020521+0000", 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038991699999999999 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023754+0000", 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003233072 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023833+0000", 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9221999999999995e-05 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.023941+0000", 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107633 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024050+0000", 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109349 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008914+0000", 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.621016921999999, 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015372158, 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 113 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008914+0000", 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008914+0000", 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008915+0000", 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.516e-06 2024-09-21T23:10:01.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008927+0000", 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2113000000000001e-05 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008949+0000", 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1353e-05 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008964+0000", 2024-09-21T23:10:01.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.577e-05 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021757+0000", 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012792112 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.021897+0000", 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014063899999999999 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.022102+0000", 2024-09-21T23:10:01.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020499799999999999 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024150+0000", 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020473779999999999 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024169+0000", 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8966000000000001e-05 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024234+0000", 2024-09-21T23:10:01.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4943000000000005e-05 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024286+0000", 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2370000000000002e-05 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.008991+0000", 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.620939276000001, 2024-09-21T23:10:01.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019578175, 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 114 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008991+0000", 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008991+0000", 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.008992+0000", 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009004+0000", 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1853e-05 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009024+0000", 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9893000000000001e-05 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009038+0000", 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4163e-05 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024360+0000", 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015321942 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024511+0000", 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015044200000000001 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.024703+0000", 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019235899999999999 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028089+0000", 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0033858439999999998 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028288+0000", 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000199075 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028429+0000", 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140975 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028570+0000", 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014050900000000001 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.671 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.009063+0000", 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.6208679, 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020902028999999999, 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 115 2024-09-21T23:10:01.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009063+0000", 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009063+0000", 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009064+0000", 2024-09-21T23:10:01.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.176e-06 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009305+0000", 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024120200000000001 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009327+0000", 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2175999999999999e-05 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009344+0000", 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6376999999999999e-05 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.025359+0000", 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016015775999999999 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.025596+0000", 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000236135 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.025972+0000", 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037593400000000002 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028246+0000", 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022745899999999999 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029655+0000", 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001408999 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029796+0000", 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014105500000000001 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029965+0000", 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016860900000000001 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.009374+0000", 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.620556289, 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020861524999999999, 2024-09-21T23:10:01.677 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 116 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009374+0000", 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009374+0000", 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009376+0000", 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.463e-06 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009387+0000", 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1463e-05 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009406+0000", 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8977000000000001e-05 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009420+0000", 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3967e-05 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.027097+0000", 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017676586000000001 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.027271+0000", 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000174659 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.027469+0000", 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000197825 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029614+0000", 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00214496 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029995+0000", 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038081399999999999 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030132+0000", 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013733199999999999 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030236+0000", 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103479 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.682 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.009444+0000", 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.620487099999998, 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021485691000000001, 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.683 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 117 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009444+0000", 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009444+0000", 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009445+0000", 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.08e-06 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009458+0000", 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.358e-05 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009478+0000", 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9323000000000001e-05 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.009497+0000", 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9239999999999999e-05 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028716+0000", 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019218981999999999 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.028904+0000", 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018812200000000001 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.029109+0000", 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020463799999999999 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030248+0000", 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011393250000000001 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030791+0000", 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000543429 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030872+0000", 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.030929+0000", 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6736e-05 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.485790+0000", 2024-09-21T23:10:01.688 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.144140278999998, 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012605139, 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 132 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485790+0000", 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485790+0000", 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485791+0000", 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6700000000000003e-07 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485796+0000", 2024-09-21T23:10:01.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485805+0000", 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5359999999999992e-06 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485809+0000", 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.16e-06 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.487798+0000", 2024-09-21T23:10:01.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019890960000000001 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.487886+0000", 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8182000000000004e-05 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488023+0000", 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013699799999999999 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490497+0000", 2024-09-21T23:10:01.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024741540000000001 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498343+0000", 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0078459570000000006 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498372+0000", 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8892000000000001e-05 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498395+0000", 2024-09-21T23:10:01.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3447000000000001e-05 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.143959028000001, 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01262986, 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 133 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485972+0000", 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6199999999999996e-07 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485977+0000", 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3880000000000002e-06 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485988+0000", 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1233000000000001e-05 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.485995+0000", 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0129999999999996e-06 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488384+0000", 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023886979999999999 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488454+0000", 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0320000000000004e-05 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.488569+0000", 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114868 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491869+0000", 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003300028 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498461+0000", 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065922000000000003 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498544+0000", 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3089000000000004e-05 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498601+0000", 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7160999999999997e-05 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.699 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.143890074000002, 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012637317, 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 134 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486041+0000", 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0400000000000005e-07 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486112+0000", 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486133+0000", 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0954e-05 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486143+0000", 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0190999999999999e-05 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489159+0000", 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030160999999999999 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489237+0000", 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8597999999999995e-05 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489356+0000", 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118837 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491876+0000", 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00251973 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498631+0000", 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0067548520000000004 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498659+0000", 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7835999999999999e-05 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498678+0000", 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9215000000000002e-05 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486191+0000", 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.143739850999999, 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012553811, 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 135 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486191+0000", 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486191+0000", 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486192+0000", 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.043e-06 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486199+0000", 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.674e-06 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486210+0000", 2024-09-21T23:10:01.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1942e-05 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486218+0000", 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0539999999999995e-06 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.489939+0000", 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037200699999999998 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490026+0000", 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7724000000000002e-05 2024-09-21T23:10:01.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490158+0000", 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013132700000000001 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493021+0000", 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002862945 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498702+0000", 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056812520000000004 2024-09-21T23:10:01.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498727+0000", 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4973999999999998e-05 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498745+0000", 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7805999999999999e-05 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486266+0000", 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.143664411, 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012596937000000001, 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 136 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486266+0000", 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486266+0000", 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486267+0000", 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0699999999999996e-07 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486274+0000", 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6109999999999999e-06 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486285+0000", 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0896999999999999e-05 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486294+0000", 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0019999999999995e-06 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490552+0000", 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042588160000000003 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490623+0000", 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0487000000000005e-05 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.490727+0000", 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103885 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493029+0000", 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023023750000000002 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498784+0000", 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005754459 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498830+0000", 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6288999999999997e-05 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498863+0000", 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3309000000000002e-05 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486310+0000", 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.14362071, 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01263121, 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.716 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 137 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486310+0000", 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486310+0000", 2024-09-21T23:10:01.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486311+0000", 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486388+0000", 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7304000000000004e-05 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486398+0000", 2024-09-21T23:10:01.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6770000000000007e-06 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486403+0000", 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.553e-06 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491135+0000", 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047316900000000002 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491221+0000", 2024-09-21T23:10:01.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5629000000000003e-05 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491353+0000", 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013197 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494512+0000", 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0031596480000000001 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498896+0000", 2024-09-21T23:10:01.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043832419999999999 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498920+0000", 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4601999999999999e-05 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498941+0000", 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1030999999999999e-05 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.143503853999999, 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012620973000000001, 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 138 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486427+0000", 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486432+0000", 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3209999999999998e-06 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486441+0000", 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7700000000000007e-06 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486445+0000", 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2699999999999998e-06 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491921+0000", 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0054762919999999998 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.491990+0000", 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9093999999999999e-05 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492091+0000", 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101072 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494518+0000", 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002426344 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.498979+0000", 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0044613919999999998 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499021+0000", 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1644999999999999e-05 2024-09-21T23:10:01.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499048+0000", 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7146000000000001e-05 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.143461026000001, 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012646737, 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.727 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 139 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:10:01.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486470+0000", 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486474+0000", 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8959999999999996e-06 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486481+0000", 2024-09-21T23:10:01.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6850000000000004e-06 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486486+0000", 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1220000000000003e-06 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492487+0000", 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060009369999999996 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492577+0000", 2024-09-21T23:10:01.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0098000000000004e-05 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.492708+0000", 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013093100000000001 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494526+0000", 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018177829999999999 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499070+0000", 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045442540000000002 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499096+0000", 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6432e-05 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499116+0000", 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0058999999999999e-05 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.14343702, 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012957738, 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 140 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486494+0000", 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7e-07 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486537+0000", 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3307999999999999e-05 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486544+0000", 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8040000000000004e-06 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486548+0000", 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3669999999999999e-06 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493081+0000", 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065326919999999997 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493151+0000", 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9716999999999997e-05 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493253+0000", 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101832 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494530+0000", 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00127761 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499404+0000", 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004873835 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499430+0000", 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5533000000000001e-05 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.499451+0000", 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1670000000000001e-05 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4210.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:09:42.486558+0000", 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.143372331999998, 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013644119, 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:01.738 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4210", 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.175:0/1235398995", 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 141 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486558+0000", 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486558+0000", 2024-09-21T23:10:01.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486559+0000", 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.75e-07 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486562+0000", 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486575+0000", 2024-09-21T23:10:01.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3122000000000001e-05 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.486587+0000", 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2087e-05 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.493990+0000", 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0074032000000000004 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494092+0000", 2024-09-21T23:10:01.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010121 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494222+0000", 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129825 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.494849+0000", 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062788000000000004 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.500155+0000", 2024-09-21T23:10:01.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053054529999999999 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.500179+0000", 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4300000000000001e-05 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:09:42.500202+0000", 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3258999999999999e-05 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:01.743 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:01.744 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:01.745 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:01.745 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:01.742+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:01.846 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:01.846+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:01.862 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:01.863 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:01.863 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:01.863 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:01.876 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:01.947 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:01.950+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:01.998 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:01.998 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:01.998 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:01.998 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:01.998 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:02.013 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:02.047 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:02.046+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:02.140 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:02.140 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:02.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:02.140 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:02.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-09-21T23:10:02.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.697884+0000", 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.440814459999999, 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041145219999999998, 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697884+0000", 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697884+0000", 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697886+0000", 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6139999999999999e-06 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697921+0000", 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5318999999999998e-05 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697951+0000", 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9366999999999999e-05 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.697974+0000", 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3093000000000001e-05 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.698116+0000", 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014264900000000001 2024-09-21T23:10:02.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.698152+0000", 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5583000000000001e-05 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.701914+0000", 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037623940000000001 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.701999+0000", 2024-09-21T23:10:02.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4503000000000001e-05 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:56 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'34, mlcod=15'29)", 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.764437+0000", 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.374261412999999, 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044487579999999997, 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764437+0000", 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764437+0000", 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764439+0000", 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.223e-06 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764448+0000", 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8200000000000003e-06 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764467+0000", 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9709999999999999e-05 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764478+0000", 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0380000000000001e-05 2024-09-21T23:10:02.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764569+0000", 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1693000000000005e-05 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.764609+0000", 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9619000000000001e-05 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.768804+0000", 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041955639999999997 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.768886+0000", 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1749000000000006e-05 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:10:02.149 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.343861027999999, 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055137770000000001, 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:10:02.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794840+0000", 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.773e-06 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794892+0000", 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2266999999999999e-05 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794921+0000", 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9493e-05 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794935+0000", 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.341e-05 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.795055+0000", 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120192 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.795106+0000", 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0875999999999999e-05 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.800255+0000", 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051487499999999997 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.800352+0000", 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7015999999999994e-05 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.153 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:71 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'45, mlcod=15'43)", 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.804538+0000", 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.334161326, 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042590070000000004, 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804538+0000", 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804538+0000", 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804539+0000", 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7430000000000001e-06 2024-09-21T23:10:02.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804593+0000", 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3720000000000001e-05 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804622+0000", 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9252999999999999e-05 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804636+0000", 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3716999999999999e-05 2024-09-21T23:10:02.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804757+0000", 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012062599999999999 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.804808+0000", 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1814999999999998e-05 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808698+0000", 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0038892570000000001 2024-09-21T23:10:02.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808797+0000", 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8876000000000007e-05 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:73 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'47, mlcod=15'43)", 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.807632+0000", 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.331067093000001, 2024-09-21T23:10:02.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004089695, 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807632+0000", 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807632+0000", 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807634+0000", 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7260000000000001e-06 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807686+0000", 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2099999999999999e-05 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807715+0000", 2024-09-21T23:10:02.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9417e-05 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807729+0000", 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3692999999999999e-05 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807854+0000", 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012536599999999999 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.807905+0000", 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0849000000000001e-05 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.811629+0000", 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037243770000000001 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.811721+0000", 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2167e-05 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:74 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'48, mlcod=15'43)", 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.329776933000002, 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005385875, 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.163 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:10:02.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808923+0000", 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3260000000000001e-06 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808945+0000", 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1557e-05 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808967+0000", 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1892999999999999e-05 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808978+0000", 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-05 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.809110+0000", 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013212200000000001 2024-09-21T23:10:02.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.809176+0000", 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6600000000000006e-05 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.814232+0000", 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050558340000000004 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.814308+0000", 2024-09-21T23:10:02.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5673000000000005e-05 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:109 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'71, mlcod=15'67)", 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.014090+0000", 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.124608929000001, 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045428869999999998, 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014090+0000", 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014090+0000", 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014091+0000", 2024-09-21T23:10:02.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6800000000000009e-07 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014155+0000", 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3707000000000005e-05 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014170+0000", 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5756e-05 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014181+0000", 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0986e-05 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014272+0000", 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0660000000000003e-05 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.014295+0000", 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3193e-05 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.018522+0000", 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042264149999999999 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.018633+0000", 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111202 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.172 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:130 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'88, mlcod=15'85)", 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.652900877, 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070102289999999998, 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485799+0000", 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.347e-06 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485864+0000", 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799000000000001e-05 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485882+0000", 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485898+0000", 2024-09-21T23:10:02.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6039999999999999e-05 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486068+0000", 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000169815 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486119+0000", 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0599000000000001e-05 2024-09-21T23:10:02.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492744+0000", 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066256759999999996 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492808+0000", 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3765999999999999e-05 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:131 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'89, mlcod=15'85)", 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.652764461, 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069196989999999996, 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485935+0000", 2024-09-21T23:10:02.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486164+0000", 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000228571 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486204+0000", 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9802999999999998e-05 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486220+0000", 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6820000000000002e-05 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486789+0000", 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00056830900000000002 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486835+0000", 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6643e-05 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492814+0000", 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059788030000000004 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492854+0000", 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.994e-05 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:132 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'90, mlcod=15'85)", 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.652428867000001, 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0099194130000000002, 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.183 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486271+0000", 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486281+0000", 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0207000000000001e-05 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486300+0000", 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9035999999999999e-05 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486313+0000", 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2986999999999999e-05 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487723+0000", 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001409532 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487774+0000", 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0966e-05 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496125+0000", 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083516130000000008 2024-09-21T23:10:02.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496189+0000", 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4048999999999996e-05 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:133 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'91, mlcod=15'85)", 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.651933278000001, 2024-09-21T23:10:02.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094674270000000005, 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486767+0000", 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.35e-06 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486779+0000", 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1847e-05 2024-09-21T23:10:02.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486800+0000", 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1413e-05 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486815+0000", 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457000000000001e-05 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488394+0000", 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001579247 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488444+0000", 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0206e-05 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496195+0000", 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077509340000000001 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496233+0000", 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7972999999999997e-05 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.651091353999998, 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087130990000000002, 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.193 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487609+0000", 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.367e-06 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487679+0000", 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9702000000000005e-05 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487702+0000", 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3626999999999999e-05 2024-09-21T23:10:02.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487722+0000", 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9366000000000001e-05 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488981+0000", 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012596300000000001 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489033+0000", 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1910000000000003e-05 2024-09-21T23:10:02.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496237+0000", 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072040580000000002 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496321+0000", 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3438999999999999e-05 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:135 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'93, mlcod=15'86)", 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.650331883, 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080451580000000002, 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488368+0000", 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3e-06 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488380+0000", 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1976e-05 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488402+0000", 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1997e-05 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488417+0000", 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4986e-05 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489990+0000", 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015723079999999999 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490021+0000", 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1479999999999997e-05 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496328+0000", 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063071409999999996 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496412+0000", 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999997e-05 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'86)", 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.649765625000001, 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075262560000000003, 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488935+0000", 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2640000000000001e-06 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488944+0000", 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488961+0000", 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7159999999999998e-05 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488972+0000", 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.042e-05 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490619+0000", 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00164788 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490651+0000", 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1869999999999998e-05 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496420+0000", 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057688230000000002 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496460+0000", 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9412999999999997e-05 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.206 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'86)", 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.649042193, 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068427269999999998, 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489658+0000", 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.703e-06 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489712+0000", 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4011999999999997e-05 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489741+0000", 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8144e-05 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489755+0000", 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4270000000000001e-05 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491279+0000", 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001523834 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491307+0000", 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8180000000000001e-05 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496464+0000", 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051571480000000003 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496499+0000", 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5435999999999998e-05 2024-09-21T23:10:02.210 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:138 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'96, mlcod=15'86)", 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.648399585, 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062358350000000003, 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490301+0000", 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.917e-06 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490315+0000", 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490344+0000", 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490361+0000", 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6883000000000001e-05 2024-09-21T23:10:02.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492263+0000", 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001902405 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492296+0000", 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2650000000000001e-05 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496504+0000", 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042075949999999997 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496535+0000", 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1506e-05 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:139 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'97, mlcod=15'86)", 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:10:02.215 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.647782719999999, 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056577900000000002, 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:10:02.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490918+0000", 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3969999999999999e-06 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490929+0000", 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1443e-05 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490953+0000", 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4023e-05 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490970+0000", 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6609999999999999e-05 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492980+0000", 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020102779999999999 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493012+0000", 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1846e-05 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496539+0000", 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035270430000000001 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496574+0000", 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5150000000000001e-05 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.219 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:140 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'98, mlcod=15'86)", 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.647161725, 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057984830000000001, 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491539+0000", 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3629999999999999e-06 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491661+0000", 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012253899999999999 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491684+0000", 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3173999999999999e-05 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491701+0000", 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7105999999999999e-05 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493648+0000", 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001946309 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493677+0000", 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9586000000000001e-05 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.497248+0000", 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035708100000000002 2024-09-21T23:10:02.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.497336+0000", 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7596000000000006e-05 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:141 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'99, mlcod=15'86)", 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.646279841999998, 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005692557, 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492420+0000", 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492429+0000", 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0629999999999995e-06 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492446+0000", 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6307000000000002e-05 2024-09-21T23:10:02.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492457+0000", 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.154e-05 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496632+0000", 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041749910000000003 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496661+0000", 2024-09-21T23:10:02.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9084000000000001e-05 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.498022+0000", 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013602589999999999 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.498112+0000", 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0053e-05 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.228 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.7588594969999995, 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063068619999999999, 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11 2024-09-21T23:10:02.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379842+0000", 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1780000000000002e-06 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379856+0000", 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4936999999999999e-05 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379878+0000", 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1248999999999999e-05 2024-09-21T23:10:02.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379888+0000", 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0302999999999999e-05 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379983+0000", 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4987000000000004e-05 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379992+0000", 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.568e-06 2024-09-21T23:10:02.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383095+0000", 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031032379999999999 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383305+0000", 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020978199999999999 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383621+0000", 2024-09-21T23:10:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000316562 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.385245+0000", 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016241459999999999 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.385925+0000", 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067988399999999996 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.386038+0000", 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113081 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.386146+0000", 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107947 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.235 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:02.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:02.237 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:02.238+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:02.338 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:02.334+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:02.362 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:02.363 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:02.363 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:02.363 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:02.376 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:02.391 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:10:02.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:10:02.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:10:02.438 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:02.438+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:02.516 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:02.516 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:02.516 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:02.516 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:02.516 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:02.517 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone1 from 0xbc63fa9 (overlap 0xa6e38c) to 0x0 2024-09-21T23:10:02.535 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:02.539 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:02.538+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:02.621 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-21T23:10:02.640 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:02.638+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:02.708 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:10:02.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:46.720844+0000", 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.986249732999999, 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022363890000000001, 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720844+0000", 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720844+0000", 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720845+0000", 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720853+0000", 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4670000000000004e-06 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720884+0000", 2024-09-21T23:10:02.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0552999999999997e-05 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.720903+0000", 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8998999999999999e-05 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.721051+0000", 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148176 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.721103+0000", 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2333000000000002e-05 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.722979+0000", 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001876445 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:46.723080+0000", 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010086600000000001 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.985764003, 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002253339, 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.714 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721330+0000", 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.75e-07 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721347+0000", 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6390999999999999e-05 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721375+0000", 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8263e-05 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721398+0000", 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2631000000000001e-05 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721547+0000", 2024-09-21T23:10:02.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014990700000000001 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.721608+0000", 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0646999999999997e-05 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.723466+0000", 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018583759999999999 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:47.723583+0000", 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116549 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3355189860000003, 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00125829, 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371575+0000", 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3300000000000003e-07 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371585+0000", 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0299000000000001e-05 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371595+0000", 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6569999999999992e-06 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371607+0000", 2024-09-21T23:10:02.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1664999999999999e-05 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371698+0000", 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1651999999999996e-05 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.371724+0000", 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5508999999999999e-05 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.372707+0000", 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00098331400000000011 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.372833+0000", 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000125761 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:540 3.1a e25/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 25'60, mlcod=25'58)", 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.349368+0000", 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.3577258859999999, 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00098529299999999993, 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349368+0000", 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349368+0000", 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349370+0000", 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2529999999999999e-06 2024-09-21T23:10:02.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349387+0000", 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6937000000000001e-05 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349411+0000", 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3682000000000001e-05 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349424+0000", 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3688e-05 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349505+0000", 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0919000000000002e-05 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.349527+0000", 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1738000000000001e-05 2024-09-21T23:10:02.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.350293+0000", 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00076552599999999999 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.350353+0000", 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0550000000000001e-05 2024-09-21T23:10:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:549 3.1a e25/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 25'61, mlcod=25'59)", 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.357537+0000", 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.34955647, 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00089526099999999995, 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.730 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357537+0000", 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357537+0000", 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357538+0000", 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2999999999999999e-07 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357544+0000", 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4480000000000002e-06 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357556+0000", 2024-09-21T23:10:02.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2982999999999999e-05 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357562+0000", 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357623+0000", 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0761999999999999e-05 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.357646+0000", 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3436000000000001e-05 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.358387+0000", 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074006099999999997 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.358432+0000", 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5954000000000001e-05 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:550 3.4 3:251a9a20:::rbd_data.11499f70651.0000000000000005:head [set-alloc-hint object_size 33554432 write_size 33554432,write 11764115~33401] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.361013+0000", 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.3460807969999999, 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015094100000000001, 2024-09-21T23:10:02.735 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 550 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361013+0000", 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361013+0000", 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361014+0000", 2024-09-21T23:10:02.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3960000000000001e-06 2024-09-21T23:10:02.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361029+0000", 2024-09-21T23:10:02.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4552999999999999e-05 2024-09-21T23:10:02.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361044+0000", 2024-09-21T23:10:02.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4859e-05 2024-09-21T23:10:02.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361056+0000", 2024-09-21T23:10:02.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1990999999999999e-05 2024-09-21T23:10:02.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361122+0000", 2024-09-21T23:10:02.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6463000000000004e-05 2024-09-21T23:10:02.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361185+0000", 2024-09-21T23:10:02.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3116999999999999e-05 2024-09-21T23:10:02.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-21T23:10:02.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361287+0000", 2024-09-21T23:10:02.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101965 2024-09-21T23:10:02.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:02.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362038+0000", 2024-09-21T23:10:02.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00075072099999999998 2024-09-21T23:10:02.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:02.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362455+0000", 2024-09-21T23:10:02.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041733599999999998 2024-09-21T23:10:02.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362490+0000", 2024-09-21T23:10:02.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4863999999999997e-05 2024-09-21T23:10:02.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362522+0000", 2024-09-21T23:10:02.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2144999999999998e-05 2024-09-21T23:10:02.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.747 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.748 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:556 3.1a e25/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 25'63, mlcod=25'61)", 2024-09-21T23:10:02.748 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.375902+0000", 2024-09-21T23:10:02.748 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.3311920660000001, 2024-09-21T23:10:02.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0009856489999999999, 2024-09-21T23:10:02.748 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.748 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.749 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.749 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.375902+0000", 2024-09-21T23:10:02.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.375902+0000", 2024-09-21T23:10:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.375903+0000", 2024-09-21T23:10:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.81e-06 2024-09-21T23:10:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.375918+0000", 2024-09-21T23:10:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4987999999999999e-05 2024-09-21T23:10:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.375944+0000", 2024-09-21T23:10:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6075e-05 2024-09-21T23:10:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.375958+0000", 2024-09-21T23:10:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3706000000000001e-05 2024-09-21T23:10:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.376027+0000", 2024-09-21T23:10:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8917000000000004e-05 2024-09-21T23:10:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.376048+0000", 2024-09-21T23:10:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0564000000000002e-05 2024-09-21T23:10:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.376851+0000", 2024-09-21T23:10:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000803007 2024-09-21T23:10:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.376887+0000", 2024-09-21T23:10:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6582000000000003e-05 2024-09-21T23:10:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:568 3.1a e25/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 25'64, mlcod=25'62)", 2024-09-21T23:10:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.394604+0000", 2024-09-21T23:10:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.312489858, 2024-09-21T23:10:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00096402300000000005, 2024-09-21T23:10:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394604+0000", 2024-09-21T23:10:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394604+0000", 2024-09-21T23:10:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394605+0000", 2024-09-21T23:10:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.53e-06 2024-09-21T23:10:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394616+0000", 2024-09-21T23:10:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1015e-05 2024-09-21T23:10:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394638+0000", 2024-09-21T23:10:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1141999999999998e-05 2024-09-21T23:10:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394647+0000", 2024-09-21T23:10:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6439999999999995e-06 2024-09-21T23:10:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394802+0000", 2024-09-21T23:10:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000154578 2024-09-21T23:10:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.394822+0000", 2024-09-21T23:10:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0259e-05 2024-09-21T23:10:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.395532+0000", 2024-09-21T23:10:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00070978299999999996 2024-09-21T23:10:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.395568+0000", 2024-09-21T23:10:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6072000000000003e-05 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:571 3.4 3:251a9a20:::rbd_data.11499f70651.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.400779+0000", 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.3063145220000001, 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014324559999999999, 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 571 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400779+0000", 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400779+0000", 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400781+0000", 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.525e-06 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400788+0000", 2024-09-21T23:10:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0779999999999998e-06 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400801+0000", 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3723000000000001e-05 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400808+0000", 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2049999999999999e-06 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400874+0000", 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5924000000000001e-05 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.400940+0000", 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6279999999999996e-05 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-21T23:10:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401018+0000", 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7985999999999995e-05 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401781+0000", 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00076348699999999998 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.402100+0000", 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031899800000000002 2024-09-21T23:10:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.402158+0000", 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7351e-05 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.402212+0000", 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3899e-05 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:572 3.1a e26/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 26'66, mlcod=25'64)", 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.403336+0000", 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.303757455, 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001446178, 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403336+0000", 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403336+0000", 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403337+0000", 2024-09-21T23:10:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.079e-06 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403343+0000", 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.242e-06 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403354+0000", 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1151e-05 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403361+0000", 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2570000000000001e-06 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403415+0000", 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3837e-05 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.403430+0000", 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5364999999999999e-05 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.404723+0000", 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012929479999999999 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.404782+0000", 2024-09-21T23:10:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9299000000000003e-05 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:575 3.1a e26/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 26'68, mlcod=26'66)", 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.408566+0000", 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.298527199, 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012920970000000001, 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408566+0000", 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408566+0000", 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408567+0000", 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408573+0000", 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.998e-06 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408586+0000", 2024-09-21T23:10:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2204e-05 2024-09-21T23:10:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408593+0000", 2024-09-21T23:10:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.554e-06 2024-09-21T23:10:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408641+0000", 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7661000000000003e-05 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.408656+0000", 2024-09-21T23:10:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5043e-05 2024-09-21T23:10:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.409829+0000", 2024-09-21T23:10:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011729939999999999 2024-09-21T23:10:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.409859+0000", 2024-09-21T23:10:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9660000000000001e-05 2024-09-21T23:10:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:595 3.1a e26/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 26'70, mlcod=26'68)", 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.426104+0000", 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.280990072, 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013318080000000001, 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426104+0000", 2024-09-21T23:10:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426104+0000", 2024-09-21T23:10:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426104+0000", 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1900000000000001e-07 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426119+0000", 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4847e-05 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426130+0000", 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1184e-05 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426138+0000", 2024-09-21T23:10:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9959999999999993e-06 2024-09-21T23:10:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426184+0000", 2024-09-21T23:10:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5036e-05 2024-09-21T23:10:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.426197+0000", 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3759e-05 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.427400+0000", 2024-09-21T23:10:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012024360000000001 2024-09-21T23:10:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.427435+0000", 2024-09-21T23:10:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5731000000000001e-05 2024-09-21T23:10:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:614 3.1a e26/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 26'72, mlcod=26'70)", 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.457504+0000", 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.2495900820000001, 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014959890000000001, 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457504+0000", 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457504+0000", 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457506+0000", 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0959999999999999e-06 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457519+0000", 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3414999999999999e-05 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457546+0000", 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6506999999999999e-05 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457556+0000", 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0458999999999999e-05 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457624+0000", 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7459000000000006e-05 2024-09-21T23:10:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.457644+0000", 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9976999999999999e-05 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.458968+0000", 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013241220000000001 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.459000+0000", 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1953999999999999e-05 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:623 3.1a e26/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 26'73, mlcod=26'71)", 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.465197+0000", 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.241896723, 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00092149300000000001, 2024-09-21T23:10:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465197+0000", 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465197+0000", 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465199+0000", 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.215e-06 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465212+0000", 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3307e-05 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465233+0000", 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0443999999999999e-05 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465252+0000", 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649000000000001e-05 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465312+0000", 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0764000000000003e-05 2024-09-21T23:10:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.465330+0000", 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7903000000000001e-05 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.466083+0000", 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00075311799999999999 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.466118+0000", 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5092999999999998e-05 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:631 3.1a e26/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 26'74, mlcod=26'72)", 2024-09-21T23:10:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.482390+0000", 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.224703989, 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001364004, 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482390+0000", 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482390+0000", 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482391+0000", 2024-09-21T23:10:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.655e-06 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482401+0000", 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1570000000000006e-06 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482419+0000", 2024-09-21T23:10:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8292999999999999e-05 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482426+0000", 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6490000000000008e-06 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482494+0000", 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7051999999999994e-05 2024-09-21T23:10:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.482511+0000", 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7812000000000002e-05 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.483714+0000", 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012021969999999999 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.483754+0000", 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0188999999999998e-05 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:662 3.1a e26/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 26'76, mlcod=26'74)", 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.523030+0000", 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.184063718, 2024-09-21T23:10:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013816200000000001, 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523030+0000", 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523030+0000", 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523031+0000", 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.364e-06 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523039+0000", 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3250000000000002e-06 2024-09-21T23:10:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523058+0000", 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9032999999999999e-05 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523065+0000", 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.233e-06 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523114+0000", 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8721000000000003e-05 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.523128+0000", 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.447e-05 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.524379+0000", 2024-09-21T23:10:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012511110000000001 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.524412+0000", 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2363000000000002e-05 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:680 3.1a e27/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 27'78, mlcod=26'76)", 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.409233+0000", 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.29786070799999997, 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010081160000000001, 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409233+0000", 2024-09-21T23:10:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409233+0000", 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409233+0000", 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3200000000000001e-07 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409238+0000", 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409254+0000", 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5517000000000001e-05 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409263+0000", 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8440000000000004e-06 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409323+0000", 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0155000000000003e-05 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.409339+0000", 2024-09-21T23:10:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5684000000000001e-05 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.410180+0000", 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00084088800000000003 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.410241+0000", 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1583999999999999e-05 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:725 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.471070+0000", 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.236023498, 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021558390000000001, 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 725 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471070+0000", 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471070+0000", 2024-09-21T23:10:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471072+0000", 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7600000000000001e-06 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471079+0000", 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3200000000000002e-06 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471096+0000", 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6849000000000001e-05 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471107+0000", 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0380999999999999e-05 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471203+0000", 2024-09-21T23:10:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6163999999999996e-05 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471422+0000", 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000219311 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471668+0000", 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024651099999999998 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.472611+0000", 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00094231800000000004 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.473101+0000", 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049056800000000004 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.473169+0000", 2024-09-21T23:10:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7892999999999995e-05 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.473226+0000", 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6764000000000001e-05 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:759 3.1a e27/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 27'79, mlcod=26'77)", 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.519944+0000", 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.187149763, 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001122051, 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.519944+0000", 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.519944+0000", 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.519946+0000", 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.519962+0000", 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5276000000000001e-05 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.519988+0000", 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5952000000000002e-05 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.520003+0000", 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5923999999999999e-05 2024-09-21T23:10:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.520091+0000", 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7321e-05 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.520120+0000", 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9251000000000002e-05 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.521008+0000", 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088774999999999998 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.521066+0000", 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8217000000000002e-05 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:766 3.1a e27/21 3:59e85b1c:::rbd_object_map.11499f70651:head v 27'80, mlcod=27'78)", 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.535797+0000", 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.17129628199999999, 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00095536000000000004, 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535797+0000", 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535797+0000", 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535800+0000", 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3029999999999998e-06 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535815+0000", 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5434000000000002e-05 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535841+0000", 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5673e-05 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535855+0000", 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4297000000000001e-05 2024-09-21T23:10:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535932+0000", 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6976000000000003e-05 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.535951+0000", 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8672999999999999e-05 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.536718+0000", 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00076709599999999999 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.536753+0000", 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4907999999999998e-05 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:02.840 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:02.841 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:02.842 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:02.843 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:02.842+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:02.931 INFO:teuthology.orchestra.run.smithi150.stdout:247 read 0x7d1d2b8 thru 0x7d26db8 (0x9b01 bytes) 2024-09-21T23:10:02.932 INFO:teuthology.orchestra.run.smithi150.stdout:248 write 0x8af6aad thru 0x8b01fba (0xb50e bytes) 2024-09-21T23:10:02.942 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:02.942+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:02.967 INFO:teuthology.orchestra.run.smithi150.stdout:249 write 0xd626690 thru 0xd6343f1 (0xdd62 bytes) 2024-09-21T23:10:02.974 INFO:teuthology.orchestra.run.smithi150.stdout:250 punch from 0x7d1a933 to 0x7d1be52, (0x151f bytes) 2024-09-21T23:10:02.974 INFO:teuthology.orchestra.run.smithi150.stdout:251 read 0xe6f54f thru 0xe79619 (0xa0cb bytes) 2024-09-21T23:10:02.975 INFO:teuthology.orchestra.run.smithi150.stdout:253 write 0x8e17285 thru 0x8e223a7 (0xb123 bytes) 2024-09-21T23:10:02.991 INFO:teuthology.orchestra.run.smithi150.stdout:254 trunc from 0xd6343f2 to 0x2a8665b 2024-09-21T23:10:03.016 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:03.016 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:03.030 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:03.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:03.046 INFO:teuthology.orchestra.run.smithi150.stdout:258 write 0x6b45e8e thru 0x6b508ae (0xaa21 bytes) 2024-09-21T23:10:03.046 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:03.042+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:03.053 INFO:teuthology.orchestra.run.smithi150.stdout:260 write 0x6975ade thru 0x697f9e5 (0x9f08 bytes) 2024-09-21T23:10:03.057 INFO:teuthology.orchestra.run.smithi150.stdout:262 punch from 0x3fdf9f3 to 0x3fed66b, (0xdc78 bytes) 2024-09-21T23:10:03.057 INFO:teuthology.orchestra.run.smithi150.stdout:264 punch from 0x1839108 to 0x183f096, (0x5f8e bytes) 2024-09-21T23:10:03.057 INFO:teuthology.orchestra.run.smithi150.stdout:265 read 0x4d3951a thru 0x4d3c7a5 (0x328c bytes) 2024-09-21T23:10:03.067 INFO:teuthology.orchestra.run.smithi150.stdout:266 trunc from 0x6b508af to 0xbc18d7b 2024-09-21T23:10:03.079 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xe89e697 2024-09-21T23:10:03.084 INFO:teuthology.orchestra.run.smithi150.stdout:270 trunc from 0xbc18d7b to 0xe89e697 2024-09-21T23:10:03.102 INFO:teuthology.orchestra.run.smithi150.stdout:271 trunc from 0xe89e697 to 0x9756ba7 2024-09-21T23:10:03.114 INFO:teuthology.orchestra.run.smithi150.stdout:273 write 0x434e9df thru 0x435a27f (0xb8a1 bytes) 2024-09-21T23:10:03.117 INFO:teuthology.orchestra.run.smithi150.stdout:274 write 0x76a4cbd thru 0x76b0e37 (0xc17b bytes) 2024-09-21T23:10:03.124 INFO:teuthology.orchestra.run.smithi150.stdout:275 write 0x312e67e thru 0x3139e5e (0xb7e1 bytes) 2024-09-21T23:10:03.144 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:03.142+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:03.147 INFO:teuthology.orchestra.run.smithi150.stdout:276 writesame 0xe23748b thru 0xe24507f data_size 0x1(0xdbf5 bytes) 2024-09-21T23:10:03.147 INFO:teuthology.orchestra.run.smithi150.stdout:277 punch from 0x6efea53 to 0x6f0c31e, (0xd8cb bytes) 2024-09-21T23:10:03.147 INFO:teuthology.orchestra.run.smithi150.stdout:278 write 0x3a1eec1 thru 0x3a2c30c (0xd44c bytes) 2024-09-21T23:10:03.151 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:03.151 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:03.151 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:03.151 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:03.151 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:03.151 INFO:teuthology.orchestra.run.smithi150.stdout:279 writesame 0x18054bd thru 0x1812148 data_size 0x6a(0xcc8c bytes) 2024-09-21T23:10:03.157 INFO:teuthology.orchestra.run.smithi150.stdout:280 punch from 0x96da20a to 0x96e7735, (0xd52b bytes) 2024-09-21T23:10:03.157 INFO:teuthology.orchestra.run.smithi150.stdout:281 read 0x289eaa9 thru 0x28ad544 (0xea9c bytes) 2024-09-21T23:10:03.159 INFO:teuthology.orchestra.run.smithi150.stdout:283 punch from 0xb2d02aa to 0xb2d9063, (0x8db9 bytes) 2024-09-21T23:10:03.159 INFO:teuthology.orchestra.run.smithi150.stdout:284 read 0xabda6ba thru 0xabe6826 (0xc16d bytes) 2024-09-21T23:10:03.159 INFO:teuthology.orchestra.run.smithi150.stdout:285 punch from 0x8a41ed5 to 0x8a4e277, (0xc3a2 bytes) 2024-09-21T23:10:03.159 INFO:teuthology.orchestra.run.smithi150.stdout:287 read 0xbc98c12 thru 0xbca0e08 (0x81f7 bytes) 2024-09-21T23:10:03.159 INFO:teuthology.orchestra.run.smithi150.stdout:289 read 0x14580b1 thru 0x14593b2 (0x1302 bytes) 2024-09-21T23:10:03.160 INFO:teuthology.orchestra.run.smithi150.stdout:290 write 0xa002fe8 thru 0xa0056c3 (0x26dc bytes) 2024-09-21T23:10:03.165 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:03.166 INFO:teuthology.orchestra.run.smithi150.stdout:293 punch from 0xaaae416 to 0xaab51ff, (0x6de9 bytes) 2024-09-21T23:10:03.178 INFO:teuthology.orchestra.run.smithi150.stdout:294 trunc from 0xe245080 to 0x819094b 2024-09-21T23:10:03.190 INFO:teuthology.orchestra.run.smithi150.stdout:295 writesame 0x7e1ab73 thru 0x7e1fc49 data_size 0x1(0x50d7 bytes) 2024-09-21T23:10:03.212 INFO:teuthology.orchestra.run.smithi150.stdout:296 read 0x2339bad thru 0x233f8d6 (0x5d2a bytes) 2024-09-21T23:10:03.213 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:03.213 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:03.213 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:03.214 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:03.214 INFO:teuthology.orchestra.run.smithi150.stdout:301 read 0xc6cac3 thru 0xc71ce7 (0x5225 bytes) 2024-09-21T23:10:03.215 INFO:teuthology.orchestra.run.smithi150.stdout:302 flatten 2024-09-21T23:10:03.228 INFO:teuthology.orchestra.run.smithi150.stdout:303 punch from 0xefc587 to 0xf05596, (0x900f bytes) 2024-09-21T23:10:03.228 INFO:teuthology.orchestra.run.smithi150.stdout:305 clone 3 order 21 su 524288 sc 6 2024-09-21T23:10:03.245 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:03.242+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:03.290 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.574596339999999, 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039594890000000001, 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716126+0000", 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:03.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716131+0000", 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716144+0000", 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2267e-05 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716158+0000", 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4731e-05 2024-09-21T23:10:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716239+0000", 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1013999999999994e-05 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716271+0000", 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1535999999999998e-05 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720016+0000", 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037447399999999999 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720084+0000", 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8813999999999995e-05 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:550 3.4 e25/21 3:251a9a20:::rbd_data.11499f70651.0000000000000005:head v 25'14, mlcod=25'12)", 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.361381+0000", 2024-09-21T23:10:03.295 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.9293406900000001, 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00094816900000000001, 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361381+0000", 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361381+0000", 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361382+0000", 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.429e-06 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361401+0000", 2024-09-21T23:10:03.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8924999999999999e-05 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361416+0000", 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5531000000000001e-05 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361424+0000", 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0620000000000003e-06 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361506+0000", 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2894999999999998e-05 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361534+0000", 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7949999999999998e-05 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362291+0000", 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00075628999999999996 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362329+0000", 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8087000000000003e-05 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:571 3.4 e26/21 3:251a9a20:::rbd_data.11499f70651.0000000000000005:head v 26'15, mlcod=25'13)", 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.401147+0000", 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.889574104, 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081959700000000003, 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401147+0000", 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401147+0000", 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401147+0000", 2024-09-21T23:10:03.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9399999999999999e-07 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401152+0000", 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4689999999999999e-06 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401167+0000", 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5087e-05 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401177+0000", 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.007e-05 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401251+0000", 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4509000000000001e-05 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401278+0000", 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6905000000000001e-05 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401912+0000", 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00063360300000000005 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.401967+0000", 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4759999999999997e-05 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:725 3.0 e27/21 3:04479396:::rbd_id.image_client.0-clone2:head v 27'2, mlcod=0'0)", 2024-09-21T23:10:03.304 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.81886428099999997, 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010753119999999999, 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471866+0000", 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6570000000000003e-06 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471881+0000", 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4684e-05 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471891+0000", 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0424999999999999e-05 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471959+0000", 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8324000000000005e-05 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471985+0000", 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5252000000000001e-05 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.472877+0000", 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00089217400000000003 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.472932+0000", 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5426000000000001e-05 2024-09-21T23:10:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:933 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'1, mlcod=0'0)", 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.33860610000000002, 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001140825, 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952116+0000", 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952303+0000", 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187238 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952317+0000", 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4348000000000001e-05 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952326+0000", 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5439999999999999e-06 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952386+0000", 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0309000000000001e-05 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952405+0000", 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9208999999999999e-05 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.953214+0000", 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00080896899999999999 2024-09-21T23:10:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.953256+0000", 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1714000000000001e-05 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:945 3.9 e27/21 3:96c84604:::rbd_data.1149b00fdffd.0000000000000006:head v 27'2, mlcod=0'0)", 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.317060234, 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001850607, 2024-09-21T23:10:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973663+0000", 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.728e-06 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973731+0000", 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7798000000000004e-05 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973749+0000", 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8703e-05 2024-09-21T23:10:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973759+0000", 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7890000000000006e-06 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973897+0000", 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138422 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973941+0000", 2024-09-21T23:10:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3192999999999998e-05 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.975430+0000", 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001489499 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.975512+0000", 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1475000000000002e-05 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:947 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'3, mlcod=27'1)", 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.31280627799999999, 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014565820000000001, 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977916+0000", 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977965+0000", 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8903000000000003e-05 2024-09-21T23:10:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977979+0000", 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4579999999999999e-05 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977987+0000", 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3579999999999997e-06 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.978125+0000", 2024-09-21T23:10:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013831600000000001 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.978170+0000", 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5346000000000003e-05 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.979284+0000", 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011134579999999999 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.979372+0000", 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7773000000000001e-05 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:949 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'4, mlcod=27'2)", 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.293727513, 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012844040000000001, 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996995+0000", 2024-09-21T23:10:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3480000000000001e-06 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997005+0000", 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4879999999999996e-06 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997036+0000", 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1080000000000001e-05 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997046+0000", 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0338e-05 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997191+0000", 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000145069 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997231+0000", 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0309000000000003e-05 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998169+0000", 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00093785299999999999 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998278+0000", 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010891900000000001 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:951 3.9 e27/21 3:96c84604:::rbd_data.1149b00fdffd.0000000000000006:head v 27'5, mlcod=27'2)", 2024-09-21T23:10:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.29264949699999998, 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011579310000000001, 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998073+0000", 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6929999999999999e-06 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998107+0000", 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.307e-05 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998132+0000", 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5420000000000001e-05 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998146+0000", 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4463e-05 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998324+0000", 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017724200000000001 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998346+0000", 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2334e-05 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.999157+0000", 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081133300000000002 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.999230+0000", 2024-09-21T23:10:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2376e-05 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:1072 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'6, mlcod=27'4)", 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.166638+0000", 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.12408335600000001, 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00096370500000000003, 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166638+0000", 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166638+0000", 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166640+0000", 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.688e-06 2024-09-21T23:10:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166667+0000", 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7892e-05 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166690+0000", 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2459999999999998e-05 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166700+0000", 2024-09-21T23:10:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0044000000000001e-05 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166763+0000", 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3058000000000005e-05 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166786+0000", 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3436999999999999e-05 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.167565+0000", 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00077828499999999998 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.167602+0000", 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6841000000000002e-05 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:1073 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'7, mlcod=27'5)", 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.108658934, 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001363263, 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182065+0000", 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4569999999999999e-06 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182082+0000", 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6736e-05 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182113+0000", 2024-09-21T23:10:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1412999999999999e-05 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182129+0000", 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5996999999999999e-05 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182268+0000", 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013877900000000001 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182311+0000", 2024-09-21T23:10:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2942999999999999e-05 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.183338+0000", 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010276619999999999 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.183426+0000", 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7275999999999996e-05 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:03.340 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:03.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:03.383 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:03.382+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:03.397 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:10:03.404 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:10:03.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:10:03.479 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:03.478+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:03.482 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:03.514 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:03.515 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:03.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:03.548 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:03.546+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:03.623 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-21T23:10:03.630 INFO:teuthology.orchestra.run.smithi150.stdout:306 trunc from 0x819094b to 0xb57a4a1 2024-09-21T23:10:03.649 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:03.650+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:03.650 INFO:teuthology.orchestra.run.smithi150.stdout:307 punch from 0x2e82091 to 0x2e82542, (0x4b1 bytes) 2024-09-21T23:10:03.650 INFO:teuthology.orchestra.run.smithi150.stdout:308 read 0x7ae408a thru 0x7ae74f4 (0x346b bytes) 2024-09-21T23:10:03.650 INFO:teuthology.orchestra.run.smithi150.stdout:309 write 0x948c4e1 thru 0x94905a4 (0x40c4 bytes) 2024-09-21T23:10:03.656 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:03.656 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:03.669 INFO:teuthology.orchestra.run.smithi150.stdout:310 trunc from 0xb57a4a1 to 0x319f5a8 2024-09-21T23:10:03.675 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:03.675 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:03.692 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:03.692 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:03.703 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:03.703 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:03.708 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:03.708 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:03.711 INFO:teuthology.orchestra.run.smithi150.stdout:312 punch from 0x2ff90d3 to 0x30014a3, (0x83d0 bytes) 2024-09-21T23:10:03.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:03.731 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:10:03.731 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:10:03.737 INFO:teuthology.orchestra.run.smithi150.stdout:313 writesame 0xdbad8b1 thru 0xdbb0989 data_size 0x29(0x30d9 bytes) 2024-09-21T23:10:03.738 INFO:teuthology.orchestra.run.smithi150.stdout:314 read 0x813f210 thru 0x814cbc4 (0xd9b5 bytes) 2024-09-21T23:10:03.738 INFO:teuthology.orchestra.run.smithi150.stdout:317 punch from 0x43f10e0 to 0x43f123b, (0x15b bytes) 2024-09-21T23:10:03.738 INFO:teuthology.orchestra.run.smithi150.stdout:318 write 0x67ae872 thru 0x67b7f5c (0x96eb bytes) 2024-09-21T23:10:03.743 INFO:teuthology.orchestra.run.smithi150.stdout:319 writesame 0xdb20411 thru 0xdb2573a data_size 0x5(0x532a bytes) 2024-09-21T23:10:03.749 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:03.746+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:03.752 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:03.753 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:03.754 INFO:teuthology.orchestra.run.smithi150.stdout:320 writesame 0xdb8c5b thru 0xdc56f5 data_size 0x11(0xca9b bytes) 2024-09-21T23:10:03.766 INFO:teuthology.orchestra.run.smithi150.stdout:321 read 0x2f93797 thru 0x2f9536b (0x1bd5 bytes) 2024-09-21T23:10:03.767 INFO:teuthology.orchestra.run.smithi150.stdout:322 punch from 0x8e40054 to 0x8e4d4b6, (0xd462 bytes) 2024-09-21T23:10:03.767 INFO:teuthology.orchestra.run.smithi150.stdout:325 writesame 0x340e3d7 thru 0x341560b data_size 0xd(0x7235 bytes) 2024-09-21T23:10:03.771 INFO:teuthology.orchestra.run.smithi150.stdout:326 writesame 0x122351 thru 0x1278af data_size 0xf(0x555f bytes) 2024-09-21T23:10:03.774 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:03.775 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:03.783 INFO:teuthology.orchestra.run.smithi150.stdout:327 write 0xbc1e91a thru 0xbc2d111 (0xe7f8 bytes) 2024-09-21T23:10:03.797 INFO:teuthology.orchestra.run.smithi150.stdout:328 trunc from 0xdbb098a to 0x7d99694 2024-09-21T23:10:03.797 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:03.798 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:03.835 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:03.835 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:03.841 INFO:teuthology.orchestra.run.smithi150.stdout:329 writesame 0xcb7e4c3 thru 0xcb7f9ea data_size 0x4(0x1528 bytes) 2024-09-21T23:10:03.841 INFO:teuthology.orchestra.run.smithi150.stdout:330 write 0x8511308 thru 0x851b626 (0xa31f bytes) 2024-09-21T23:10:03.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:10:03.848 INFO:teuthology.orchestra.run.smithi150.stdout:331 read 0x3265e79 thru 0x3268ebb (0x3043 bytes) 2024-09-21T23:10:03.850 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:03.850+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:03.860 INFO:teuthology.orchestra.run.smithi150.stdout:332 write 0xce87560 thru 0xce88022 (0xac3 bytes) 2024-09-21T23:10:03.866 INFO:teuthology.orchestra.run.smithi150.stdout:334 read 0x1382d3e thru 0x1389132 (0x63f5 bytes) 2024-09-21T23:10:03.866 INFO:teuthology.orchestra.run.smithi150.stdout:335 read 0x9fa86a7 thru 0x9fa9295 (0xbef bytes) 2024-09-21T23:10:03.866 INFO:teuthology.orchestra.run.smithi150.stdout:339 write 0x98b3513 thru 0x98bbced (0x87db bytes) 2024-09-21T23:10:03.872 INFO:teuthology.orchestra.run.smithi150.stdout:340 writesame 0x36ebbfa thru 0x36f80b1 data_size 0x14(0xc4b8 bytes) 2024-09-21T23:10:03.875 INFO:teuthology.orchestra.run.smithi150.stdout:341 writesame 0x49b518c thru 0x49bcc11 data_size 0x1(0x7a86 bytes) 2024-09-21T23:10:03.897 INFO:teuthology.orchestra.run.smithi150.stdout:342 punch from 0xc7ec17 to 0xc8289f, (0x3c88 bytes) 2024-09-21T23:10:03.898 INFO:teuthology.orchestra.run.smithi150.stdout:343 punch from 0x18c2ae to 0x198763, (0xc4b5 bytes) 2024-09-21T23:10:03.898 INFO:teuthology.orchestra.run.smithi150.stdout:344 clone 4 order 22 su 65536 sc 8 2024-09-21T23:10:03.951 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:03.950+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:03.984 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:03.984 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:03.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:10:04.051 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:04.050+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:04.119 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:04.119 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:04.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:10:04.152 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:04.150+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:04.253 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:04.250+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:04.304 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:04.304 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:04.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:04.361 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:04.362+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:04.406 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:10:04.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:10:04.446 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:04.446 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:04.454 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:04.454+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:04.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:04.522 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone3 from 0xce88023 (overlap 0x319f5a8) to 0x1dc6031 2024-09-21T23:10:04.555 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:04.554+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:04.638 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:04.638 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:04.652 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:04.672 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-21T23:10:04.693 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:04.694+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:04.757 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:04.754+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:04.811 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:04.811 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:04.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:04.873 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:04.874+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:04.907 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:04.907 INFO:teuthology.orchestra.run.smithi150.stdout:347 write 0x97f0cb2 thru 0x97f404d (0x339c bytes) 2024-09-21T23:10:04.922 INFO:teuthology.orchestra.run.smithi150.stdout:349 write 0xa82c326 thru 0xa83bfe7 (0xfcc2 bytes) 2024-09-21T23:10:04.931 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:04.932 INFO:teuthology.orchestra.run.smithi150.stdout:353 punch from 0xbeb8d9e to 0xbec1602, (0x8864 bytes) 2024-09-21T23:10:04.932 INFO:teuthology.orchestra.run.smithi150.stdout:354 read 0x1f2d1ac thru 0x1f3248f (0x52e4 bytes) 2024-09-21T23:10:04.933 INFO:teuthology.orchestra.run.smithi150.stdout:355 write 0x3a149c2 thru 0x3a17084 (0x26c3 bytes) 2024-09-21T23:10:04.957 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:04.957 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:04.958 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:04.954+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:04.960 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:04.973 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:04.975 INFO:teuthology.orchestra.run.smithi150.stdout:358 write 0xe179d59 thru 0xe17aadb (0xd83 bytes) 2024-09-21T23:10:04.980 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:04.980 INFO:teuthology.orchestra.run.smithi150.stdout:360 read 0x9b0c5ca thru 0x9b1b098 (0xeacf bytes) 2024-09-21T23:10:04.980 INFO:teuthology.orchestra.run.smithi150.stdout:361 write 0xcc3dac9 thru 0xcc41325 (0x385d bytes) 2024-09-21T23:10:04.990 INFO:teuthology.orchestra.run.smithi150.stdout:362 read 0xac91ceb thru 0xac9b53c (0x9852 bytes) 2024-09-21T23:10:04.990 INFO:teuthology.orchestra.run.smithi150.stdout:363 read 0x936bb58 thru 0x937ae60 (0xf309 bytes) 2024-09-21T23:10:04.992 INFO:teuthology.orchestra.run.smithi150.stdout:366 read 0x5bd4286 thru 0x5be3841 (0xf5bc bytes) 2024-09-21T23:10:04.992 INFO:teuthology.orchestra.run.smithi150.stdout:367 writesame 0x91e30dd thru 0x91eff28 data_size 0xa2(0xce4c bytes) 2024-09-21T23:10:05.002 INFO:teuthology.orchestra.run.smithi150.stdout:368 write 0x96b81bb thru 0x96c1798 (0x95de bytes) 2024-09-21T23:10:05.014 INFO:teuthology.orchestra.run.smithi150.stdout:369 writesame 0xcd7aced thru 0xcd85689 data_size 0x1(0xa99d bytes) 2024-09-21T23:10:05.052 INFO:teuthology.orchestra.run.smithi150.stdout:370 read 0x3d2f7d4 thru 0x3d31826 (0x2053 bytes) 2024-09-21T23:10:05.052 INFO:teuthology.orchestra.run.smithi150.stdout:371 read 0x551adc3 thru 0x552363f (0x887d bytes) 2024-09-21T23:10:05.061 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:05.062+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:05.081 INFO:teuthology.orchestra.run.smithi150.stdout:372 trunc from 0xe17aadc to 0x318b43e 2024-09-21T23:10:05.157 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:05.157 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:05.159 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:05.158+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:05.168 INFO:teuthology.orchestra.run.smithi150.stdout:373 punch from 0x1923db5 to 0x1930998, (0xcbe3 bytes) 2024-09-21T23:10:05.168 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.168 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.173 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:05.201 INFO:teuthology.orchestra.run.smithi150.stdout:380 writesame 0xc52d7ea thru 0xc539a8f data_size 0x42(0xc2a6 bytes) 2024-09-21T23:10:05.202 INFO:teuthology.orchestra.run.smithi150.stdout:381 writesame 0x7e0217e thru 0x7e0b2b0 data_size 0x1(0x9133 bytes) 2024-09-21T23:10:05.227 INFO:teuthology.orchestra.run.smithi150.stdout:382 read 0x59474a thru 0x599087 (0x493e bytes) 2024-09-21T23:10:05.228 INFO:teuthology.orchestra.run.smithi150.stdout:385 write 0x551109a thru 0x55205e8 (0xf54f bytes) 2024-09-21T23:10:05.237 INFO:teuthology.orchestra.run.smithi150.stdout:387 write 0x7065122 thru 0x706db40 (0x8a1f bytes) 2024-09-21T23:10:05.242 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.246 INFO:teuthology.orchestra.run.smithi150.stdout:389 trunc from 0xc539a90 to 0xdeffbc5 2024-09-21T23:10:05.256 INFO:teuthology.orchestra.run.smithi150.stdout:391 read 0x5f496f4 thru 0x5f5144c (0x7d59 bytes) 2024-09-21T23:10:05.256 INFO:teuthology.orchestra.run.smithi150.stdout:392 read 0x6d7491c thru 0x6d788dd (0x3fc2 bytes) 2024-09-21T23:10:05.256 INFO:teuthology.orchestra.run.smithi150.stdout:394 read 0xa0c5869 thru 0xa0c5faf (0x747 bytes) 2024-09-21T23:10:05.256 INFO:teuthology.orchestra.run.smithi150.stdout:395 writesame 0x9993705 thru 0x9994a25 data_size 0x1(0x1321 bytes) 2024-09-21T23:10:05.260 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:05.258+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:05.272 INFO:teuthology.orchestra.run.smithi150.stdout:396 write 0xe53eeb9 thru 0xe53fc68 (0xdb0 bytes) 2024-09-21T23:10:05.277 INFO:teuthology.orchestra.run.smithi150.stdout:399 read 0x7436ee7 thru 0x74429c1 (0xbadb bytes) 2024-09-21T23:10:05.298 INFO:teuthology.orchestra.run.smithi150.stdout:400 trunc from 0xe53fc69 to 0x6bc8ac1 2024-09-21T23:10:05.317 INFO:teuthology.orchestra.run.smithi150.stdout:401 read 0x48a0d02 thru 0x48aa278 (0x9577 bytes) 2024-09-21T23:10:05.319 INFO:teuthology.orchestra.run.smithi150.stdout:403 trunc from 0x6bc8ac1 to 0x70e4e8d 2024-09-21T23:10:05.347 INFO:teuthology.orchestra.run.smithi150.stdout:404 writesame 0x89e0456 thru 0x89e4689 data_size 0x26(0x4234 bytes) 2024-09-21T23:10:05.347 INFO:teuthology.orchestra.run.smithi150.stdout:405 read 0x78e2c13 thru 0x78efc4f (0xd03d bytes) 2024-09-21T23:10:05.348 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:05.348 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:05.362 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:05.358+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:05.362 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:10:05.365 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:05.371 INFO:teuthology.orchestra.run.smithi150.stdout:406 write 0xbd5954a thru 0xbd5d6bc (0x4173 bytes) 2024-09-21T23:10:05.378 INFO:teuthology.orchestra.run.smithi150.stdout:408 writesame 0x7f2cccb thru 0x7f3acd3 data_size 0x1(0xe009 bytes) 2024-09-21T23:10:05.386 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T23:10:05.428 INFO:teuthology.orchestra.run.smithi150.stdout:411 write 0xc278bd8 thru 0xc2817b4 (0x8bdd bytes) 2024-09-21T23:10:05.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:10:05.434 INFO:teuthology.orchestra.run.smithi150.stdout:412 read 0x3ba5eda thru 0x3bafdcf (0x9ef6 bytes) 2024-09-21T23:10:05.434 INFO:teuthology.orchestra.run.smithi150.stdout:413 write 0x4cc1c29 thru 0x4cc8f85 (0x735d bytes) 2024-09-21T23:10:05.438 INFO:teuthology.orchestra.run.smithi150.stdout:414 punch from 0x697a4d8 to 0x6982104, (0x7c2c bytes) 2024-09-21T23:10:05.450 INFO:teuthology.orchestra.run.smithi150.stdout:417 trunc from 0xc2817b5 to 0x50ed530 2024-09-21T23:10:05.462 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:05.462+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:05.470 INFO:teuthology.orchestra.run.smithi150.stdout:418 read 0x338911d thru 0x338e9bc (0x58a0 bytes) 2024-09-21T23:10:05.470 INFO:teuthology.orchestra.run.smithi150.stdout:419 writesame 0x4e5a7d thru 0x4e9235 data_size 0xf(0x37b9 bytes) 2024-09-21T23:10:05.501 INFO:teuthology.orchestra.run.smithi150.stdout:420 read 0x1e5c454 thru 0x1e6b1bf (0xed6c bytes) 2024-09-21T23:10:05.502 INFO:teuthology.orchestra.run.smithi150.stdout:421 write 0x74a6de thru 0x752d7f (0x86a2 bytes) 2024-09-21T23:10:05.507 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:05.507 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:05.527 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:05.551 INFO:teuthology.orchestra.run.smithi150.stdout:423 trunc from 0x50ed530 to 0x8ddad6e 2024-09-21T23:10:05.559 INFO:teuthology.orchestra.run.smithi150.stdout:425 punch from 0xd63ac2 to 0xd6d888, (0x9dc6 bytes) 2024-09-21T23:10:05.563 INFO:teuthology.orchestra.run.smithi150.stdout:426 trunc from 0x8ddad6e to 0xabedf6d 2024-09-21T23:10:05.569 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:05.570+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:05.570 INFO:teuthology.orchestra.run.smithi150.stdout:427 write 0x1386924 thru 0x13957f1 (0xeece bytes) 2024-09-21T23:10:05.609 INFO:teuthology.orchestra.run.smithi150.stdout:428 punch from 0x9b4bb2d to 0x9b4bc6f, (0x142 bytes) 2024-09-21T23:10:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:429 write 0xd0af130 thru 0xd0b80bf (0x8f90 bytes) 2024-09-21T23:10:05.631 INFO:teuthology.orchestra.run.smithi150.stdout:430 write 0xc6cc787 thru 0xc6d2a09 (0x6283 bytes) 2024-09-21T23:10:05.638 INFO:teuthology.orchestra.run.smithi150.stdout:431 read 0x4b04e1a thru 0x4b07943 (0x2b2a bytes) 2024-09-21T23:10:05.638 INFO:teuthology.orchestra.run.smithi150.stdout:433 read 0xb48f674 thru 0xb49b97d (0xc30a bytes) 2024-09-21T23:10:05.638 INFO:teuthology.orchestra.run.smithi150.stdout:434 write 0x255d7dc thru 0x25683bd (0xabe2 bytes) 2024-09-21T23:10:05.641 INFO:teuthology.orchestra.run.smithi150.stdout:435 punch from 0x829da36 to 0x82a06c0, (0x2c8a bytes) 2024-09-21T23:10:05.641 INFO:teuthology.orchestra.run.smithi150.stdout:436 write 0x7d4e25d thru 0x7d50032 (0x1dd6 bytes) 2024-09-21T23:10:05.647 INFO:teuthology.orchestra.run.smithi150.stdout:437 punch from 0x615cc60 to 0x6160dcf, (0x416f bytes) 2024-09-21T23:10:05.647 INFO:teuthology.orchestra.run.smithi150.stdout:439 read 0x1106f6a thru 0x11108e5 (0x997c bytes) 2024-09-21T23:10:05.659 INFO:teuthology.orchestra.run.smithi150.stdout:440 trunc from 0xd0b80c0 to 0x68ca751 2024-09-21T23:10:05.664 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:05.662+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:05.685 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.685 INFO:teuthology.orchestra.run.smithi150.stdout:444 write 0x2cbcb2e thru 0x2cc23cb (0x589e bytes) 2024-09-21T23:10:05.687 INFO:teuthology.orchestra.run.smithi150.stdout:445 punch from 0x148fa8e to 0x149cb99, (0xd10b bytes) 2024-09-21T23:10:05.687 INFO:teuthology.orchestra.run.smithi150.stdout:446 read 0x57566f7 thru 0x575aedd (0x47e7 bytes) 2024-09-21T23:10:05.687 INFO:teuthology.orchestra.run.smithi150.stdout:448 write 0x3a8e2f5 thru 0x3a96b7d (0x8889 bytes) 2024-09-21T23:10:05.690 INFO:teuthology.orchestra.run.smithi150.stdout:450 read 0x4d3a003 thru 0x4d4924c (0xf24a bytes) 2024-09-21T23:10:05.691 INFO:teuthology.orchestra.run.smithi150.stdout:451 punch from 0x47620ed to 0x4766bf3, (0x4b06 bytes) 2024-09-21T23:10:05.694 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:05.694 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:05.705 INFO:teuthology.orchestra.run.smithi150.stdout:452 write 0x7689b29 thru 0x7692d70 (0x9248 bytes) 2024-09-21T23:10:05.709 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:05.712 INFO:teuthology.orchestra.run.smithi150.stdout:455 read 0x3fec778 thru 0x3ffc052 (0xf8db bytes) 2024-09-21T23:10:05.713 INFO:teuthology.orchestra.run.smithi150.stdout:457 punch from 0x356686c to 0x35736f0, (0xce84 bytes) 2024-09-21T23:10:05.713 INFO:teuthology.orchestra.run.smithi150.stdout:458 read 0x6267ab7 thru 0x6268ba5 (0x10ef bytes) 2024-09-21T23:10:05.723 INFO:teuthology.orchestra.run.smithi150.stdout:459 write 0x790f491 thru 0x79139ec (0x455c bytes) 2024-09-21T23:10:05.726 INFO:teuthology.orchestra.run.smithi150.stdout:460 read 0xf0e37f thru 0xf11945 (0x35c7 bytes) 2024-09-21T23:10:05.734 INFO:teuthology.orchestra.run.smithi150.stdout:464 trunc from 0x79139ed to 0x3e4fa8f 2024-09-21T23:10:05.761 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.761 INFO:teuthology.orchestra.run.smithi150.stdout:467 read 0xf1efeb thru 0xf2b67a (0xc690 bytes) 2024-09-21T23:10:05.762 INFO:teuthology.orchestra.run.smithi150.stdout:468 read 0x1818e74 thru 0x1820f74 (0x8101 bytes) 2024-09-21T23:10:05.763 INFO:teuthology.orchestra.run.smithi150.stdout:469 read 0x3afa181 thru 0x3afff56 (0x5dd6 bytes) 2024-09-21T23:10:05.785 INFO:teuthology.orchestra.run.smithi150.stdout:470 write 0x9a6b2dd thru 0x9a7a867 (0xf58b bytes) 2024-09-21T23:10:05.797 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:05.798+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:05.813 INFO:teuthology.orchestra.run.smithi150.stdout:471 write 0xe8621b5 thru 0xe864921 (0x276d bytes) 2024-09-21T23:10:05.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:05.818 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.818 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.818 INFO:teuthology.orchestra.run.smithi150.stdout:475 read 0x60d9ad3 thru 0x60e597b (0xbea9 bytes) 2024-09-21T23:10:05.818 INFO:teuthology.orchestra.run.smithi150.stdout:476 read 0x5543db9 thru 0x5552aec (0xed34 bytes) 2024-09-21T23:10:05.831 INFO:teuthology.orchestra.run.smithi150.stdout:477 trunc from 0xe864922 to 0x7465052 2024-09-21T23:10:05.848 INFO:teuthology.orchestra.run.smithi150.stdout:478 write 0x56a37af thru 0x56a6f63 (0x37b5 bytes) 2024-09-21T23:10:05.852 INFO:teuthology.orchestra.run.smithi150.stdout:479 write 0x396978 thru 0x3a05c9 (0x9c52 bytes) 2024-09-21T23:10:05.865 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:05.866+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:05.880 INFO:teuthology.orchestra.run.smithi150.stdout:480 read 0x5e0d637 thru 0x5e1bfaf (0xe979 bytes) 2024-09-21T23:10:05.880 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.887 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:05.888 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:05.894 INFO:teuthology.orchestra.run.smithi150.stdout:483 writesame 0x92aa4d0 thru 0x92b6d7c data_size 0x29(0xc8ad bytes) 2024-09-21T23:10:05.894 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.894 INFO:teuthology.orchestra.run.smithi150.stdout:485 punch from 0x5478357 to 0x5478964, (0x60d bytes) 2024-09-21T23:10:05.895 INFO:teuthology.orchestra.run.smithi150.stdout:486 writesame 0x389af3 thru 0x3955cd data_size 0xf(0xbadb bytes) 2024-09-21T23:10:05.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:05.908 INFO:teuthology.orchestra.run.smithi150.stdout:487 trunc from 0x92b6d7d to 0x3c94df6 2024-09-21T23:10:05.929 INFO:teuthology.orchestra.run.smithi150.stdout:488 read 0xdb758f thru 0xdc614a (0xebbc bytes) 2024-09-21T23:10:05.930 INFO:teuthology.orchestra.run.smithi150.stdout:489 read 0x25edc78 thru 0x25fbb18 (0xdea1 bytes) 2024-09-21T23:10:05.941 INFO:teuthology.orchestra.run.smithi150.stdout:490 write 0x5fd1b7e thru 0x5fd3613 (0x1a96 bytes) 2024-09-21T23:10:05.947 INFO:teuthology.orchestra.run.smithi150.stdout:492 trunc from 0x5fd3614 to 0x6e4c94e 2024-09-21T23:10:05.954 INFO:teuthology.orchestra.run.smithi150.stdout:493 punch from 0x5826c5d to 0x582c29c, (0x563f bytes) 2024-09-21T23:10:05.965 INFO:teuthology.orchestra.run.smithi150.stdout:495 trunc from 0x6e4c94e to 0xe1aeeba 2024-09-21T23:10:05.966 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:05.962+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:05.977 INFO:teuthology.orchestra.run.smithi150.stdout:496 read 0x1593ffe thru 0x159c4e9 (0x84ec bytes) 2024-09-21T23:10:05.978 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.978 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.978 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:05.978 INFO:teuthology.orchestra.run.smithi150.stdout:501 read 0xb2c0819 thru 0xb2ce0a9 (0xd891 bytes) 2024-09-21T23:10:05.979 INFO:teuthology.orchestra.run.smithi150.stdout:502 writesame 0x39974b0 thru 0x39a08e9 data_size 0x1(0x943a bytes) 2024-09-21T23:10:05.986 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:05.987 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:05.998 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.005 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:06.006 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:06.014 INFO:teuthology.orchestra.run.smithi150.stdout:505 trunc from 0xe1aeeba to 0x3237501 2024-09-21T23:10:06.024 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:06.024 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:06.040 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:06.040 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:06.040 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:06.040 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:06.042 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:06.042 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:06.043 INFO:teuthology.orchestra.run.smithi150.stdout:506 read 0x8839a1 thru 0x88cdc0 (0x9420 bytes) 2024-09-21T23:10:06.044 INFO:teuthology.orchestra.run.smithi150.stdout:507 punch from 0x8626aa to 0x86625f, (0x3bb5 bytes) 2024-09-21T23:10:06.044 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.063 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:10:06.063 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:10:06.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:06.066 INFO:teuthology.orchestra.run.smithi150.stdout:509 write 0x924383c thru 0x924938d (0x5b52 bytes) 2024-09-21T23:10:06.067 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:06.066+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:06.072 INFO:teuthology.orchestra.run.smithi150.stdout:512 write 0x4a07201 thru 0x4a0d120 (0x5f20 bytes) 2024-09-21T23:10:06.084 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:06.085 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:06.097 INFO:teuthology.orchestra.run.smithi150.stdout:513 writesame 0xd2c4a68 thru 0xd2c8106 data_size 0x3b(0x369f bytes) 2024-09-21T23:10:06.097 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.097 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.097 INFO:teuthology.orchestra.run.smithi150.stdout:516 writesame 0xbcf11ae thru 0xbcf18aa data_size 0x1(0x6fd bytes) 2024-09-21T23:10:06.103 INFO:teuthology.orchestra.run.smithi150.stdout:517 write 0x4a58936 thru 0x4a5b109 (0x27d4 bytes) 2024-09-21T23:10:06.109 INFO:teuthology.orchestra.run.smithi150.stdout:518 punch from 0xcadb9a8 to 0xcae0503, (0x4b5b bytes) 2024-09-21T23:10:06.120 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:06.121 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:06.124 INFO:teuthology.orchestra.run.smithi150.stdout:519 trunc from 0xd2c8107 to 0x29d96a1 2024-09-21T23:10:06.151 INFO:teuthology.orchestra.run.smithi150.stdout:520 read 0x2302962 thru 0x230d72c (0xadcb bytes) 2024-09-21T23:10:06.156 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:06.156 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:06.164 INFO:teuthology.orchestra.run.smithi150.stdout:521 trunc from 0x29d96a1 to 0xb884ed7 2024-09-21T23:10:06.167 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:06.166+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:06.175 INFO:teuthology.orchestra.run.smithi150.stdout:522 punch from 0x360f84e to 0x3612746, (0x2ef8 bytes) 2024-09-21T23:10:06.175 INFO:teuthology.orchestra.run.smithi150.stdout:524 read 0x197220b thru 0x197d2bc (0xb0b2 bytes) 2024-09-21T23:10:06.176 INFO:teuthology.orchestra.run.smithi150.stdout:525 punch from 0x4f48f05 to 0x4f54d49, (0xbe44 bytes) 2024-09-21T23:10:06.176 INFO:teuthology.orchestra.run.smithi150.stdout:527 punch from 0xccbbc0 to 0xccd885, (0x1cc5 bytes) 2024-09-21T23:10:06.176 INFO:teuthology.orchestra.run.smithi150.stdout:528 write 0xa117944 thru 0xa124281 (0xc93e bytes) 2024-09-21T23:10:06.186 INFO:teuthology.orchestra.run.smithi150.stdout:530 writesame 0x7eaa038 thru 0x7eb57e7 data_size 0x8(0xb7b0 bytes) 2024-09-21T23:10:06.198 INFO:teuthology.orchestra.run.smithi150.stdout:531 read 0x2f9429 thru 0x3049ac (0xb584 bytes) 2024-09-21T23:10:06.199 INFO:teuthology.orchestra.run.smithi150.stdout:532 read 0x60c7db6 thru 0x60cffb3 (0x81fe bytes) 2024-09-21T23:10:06.199 INFO:teuthology.orchestra.run.smithi150.stdout:533 punch from 0x54e820b to 0x54edeab, (0x5ca0 bytes) 2024-09-21T23:10:06.199 INFO:teuthology.orchestra.run.smithi150.stdout:534 writesame 0x914ee8f thru 0x9151349 data_size 0x1(0x24bb bytes) 2024-09-21T23:10:06.208 INFO:teuthology.orchestra.run.smithi150.stdout:536 write 0x8bcff02 thru 0x8bd243a (0x2539 bytes) 2024-09-21T23:10:06.211 INFO:teuthology.orchestra.run.smithi150.stdout:537 write 0x2fc908d thru 0x2fd10e6 (0x805a bytes) 2024-09-21T23:10:06.212 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:06.212 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:06.212 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:06.212 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:06.212 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:06.213 INFO:teuthology.orchestra.run.smithi150.stdout:538 read 0x9be547a thru 0x9be6ff6 (0x1b7d bytes) 2024-09-21T23:10:06.227 INFO:teuthology.orchestra.run.smithi150.stdout:540 writesame 0xc84d27a thru 0xc855ed8 data_size 0x1(0x8c5f bytes) 2024-09-21T23:10:06.227 INFO:teuthology.orchestra.run.smithi150.stdout:541 read 0x4950de1 thru 0x495f199 (0xe3b9 bytes) 2024-09-21T23:10:06.227 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.227 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.227 INFO:teuthology.orchestra.run.smithi150.stdout:545 write 0xb24d987 thru 0xb2569d7 (0x9051 bytes) 2024-09-21T23:10:06.228 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:06.237 INFO:teuthology.orchestra.run.smithi150.stdout:546 trunc from 0xc855ed9 to 0xc0c7a0d 2024-09-21T23:10:06.249 INFO:teuthology.orchestra.run.smithi150.stdout:547 punch from 0xbd8dcee to 0xbd90559, (0x286b bytes) 2024-09-21T23:10:06.249 INFO:teuthology.orchestra.run.smithi150.stdout:548 read 0x110c675 thru 0x1112a40 (0x63cc bytes) 2024-09-21T23:10:06.252 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.252 INFO:teuthology.orchestra.run.smithi150.stdout:551 punch from 0xa741bf3 to 0xa743f4e, (0x235b bytes) 2024-09-21T23:10:06.252 INFO:teuthology.orchestra.run.smithi150.stdout:552 write 0x5a69f73 thru 0x5a73f41 (0x9fcf bytes) 2024-09-21T23:10:06.260 INFO:teuthology.orchestra.run.smithi150.stdout:554 write 0x15186dd thru 0x151ea3f (0x6363 bytes) 2024-09-21T23:10:06.266 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.268 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:06.266+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:06.272 INFO:teuthology.orchestra.run.smithi150.stdout:557 trunc from 0xc0c7a0d to 0x7f20d2a 2024-09-21T23:10:06.299 INFO:teuthology.orchestra.run.smithi150.stdout:558 punch from 0x16b634 to 0x16ffd7, (0x49a3 bytes) 2024-09-21T23:10:06.304 INFO:teuthology.orchestra.run.smithi150.stdout:559 trunc from 0x7f20d2a to 0x4d4b053 2024-09-21T23:10:06.331 INFO:teuthology.orchestra.run.smithi150.stdout:562 writesame 0x2c930f0 thru 0x2c96add data_size 0x5(0x39ee bytes) 2024-09-21T23:10:06.334 INFO:teuthology.orchestra.run.smithi150.stdout:563 read 0x1165c29 thru 0x116d192 (0x756a bytes) 2024-09-21T23:10:06.336 INFO:teuthology.orchestra.run.smithi150.stdout:564 writesame 0xb10965 thru 0xb1717e data_size 0x52(0x681a bytes) 2024-09-21T23:10:06.341 INFO:teuthology.orchestra.run.smithi150.stdout:566 read 0x4cc00b2 thru 0x4cc9250 (0x919f bytes) 2024-09-21T23:10:06.341 INFO:teuthology.orchestra.run.smithi150.stdout:567 read 0x3e9f196 thru 0x3ea9f4d (0xadb8 bytes) 2024-09-21T23:10:06.356 INFO:teuthology.orchestra.run.smithi150.stdout:568 write 0x585f9ff thru 0x58661f3 (0x67f5 bytes) 2024-09-21T23:10:06.364 INFO:teuthology.orchestra.run.smithi150.stdout:569 writesame 0x565286d thru 0x565e90b data_size 0x3(0xc09f bytes) 2024-09-21T23:10:06.365 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:06.365 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.568242605000002, 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055137770000000001, 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794838+0000", 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794840+0000", 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.773e-06 2024-09-21T23:10:06.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794892+0000", 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2266999999999999e-05 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794921+0000", 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9493e-05 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.794935+0000", 2024-09-21T23:10:06.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.341e-05 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.795055+0000", 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120192 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.795106+0000", 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0875999999999999e-05 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.800255+0000", 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051487499999999997 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.800352+0000", 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7015999999999994e-05 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:74 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'48, mlcod=15'43)", 2024-09-21T23:10:06.370 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.554158510000001, 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005385875, 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808922+0000", 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808923+0000", 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3260000000000001e-06 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808945+0000", 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1557e-05 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808967+0000", 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1892999999999999e-05 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.808978+0000", 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-05 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.809110+0000", 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013212200000000001 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.809176+0000", 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6600000000000006e-05 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.814232+0000", 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050558340000000004 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:41.814308+0000", 2024-09-21T23:10:06.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5673000000000005e-05 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:130 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'88, mlcod=15'85)", 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.877282453999999, 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070102289999999998, 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485798+0000", 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485799+0000", 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.347e-06 2024-09-21T23:10:06.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485864+0000", 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799000000000001e-05 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485882+0000", 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485898+0000", 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6039999999999999e-05 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486068+0000", 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000169815 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486119+0000", 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0599000000000001e-05 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492744+0000", 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066256759999999996 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492808+0000", 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3765999999999999e-05 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:131 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'89, mlcod=15'85)", 2024-09-21T23:10:06.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.877146037999999, 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069196989999999996, 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485934+0000", 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.485935+0000", 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486164+0000", 2024-09-21T23:10:06.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000228571 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486204+0000", 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9802999999999998e-05 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486220+0000", 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6820000000000002e-05 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486789+0000", 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00056830900000000002 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486835+0000", 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6643e-05 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492814+0000", 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059788030000000004 2024-09-21T23:10:06.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492854+0000", 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.994e-05 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:132 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'90, mlcod=15'85)", 2024-09-21T23:10:06.384 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.876810444, 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0099194130000000002, 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486270+0000", 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486271+0000", 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486281+0000", 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0207000000000001e-05 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486300+0000", 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9035999999999999e-05 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486313+0000", 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2986999999999999e-05 2024-09-21T23:10:06.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487723+0000", 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001409532 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487774+0000", 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0966e-05 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496125+0000", 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083516130000000008 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496189+0000", 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4048999999999996e-05 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:133 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'91, mlcod=15'85)", 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.876314855, 2024-09-21T23:10:06.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094674270000000005, 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486766+0000", 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486767+0000", 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.35e-06 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486779+0000", 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1847e-05 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486800+0000", 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1413e-05 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.486815+0000", 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457000000000001e-05 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488394+0000", 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001579247 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488444+0000", 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0206e-05 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496195+0000", 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077509340000000001 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496233+0000", 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7972999999999997e-05 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.875472931000001, 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0087130990000000002, 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487608+0000", 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487609+0000", 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.367e-06 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487679+0000", 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9702000000000005e-05 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487702+0000", 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3626999999999999e-05 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.487722+0000", 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9366000000000001e-05 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488981+0000", 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012596300000000001 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489033+0000", 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1910000000000003e-05 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496237+0000", 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072040580000000002 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496321+0000", 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3438999999999999e-05 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:135 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'93, mlcod=15'86)", 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.874713459999999, 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080451580000000002, 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488367+0000", 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488368+0000", 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3e-06 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488380+0000", 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1976e-05 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488402+0000", 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1997e-05 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488417+0000", 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4986e-05 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489990+0000", 2024-09-21T23:10:06.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015723079999999999 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490021+0000", 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1479999999999997e-05 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496328+0000", 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063071409999999996 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496412+0000", 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3969999999999997e-05 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'86)", 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.874147202, 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075262560000000003, 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488933+0000", 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488935+0000", 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2640000000000001e-06 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488944+0000", 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488961+0000", 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7159999999999998e-05 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.488972+0000", 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.042e-05 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490619+0000", 2024-09-21T23:10:06.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00164788 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490651+0000", 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1869999999999998e-05 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496420+0000", 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057688230000000002 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496460+0000", 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9412999999999997e-05 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'86)", 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.873423769999999, 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068427269999999998, 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.408 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489657+0000", 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489658+0000", 2024-09-21T23:10:06.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.703e-06 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489712+0000", 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4011999999999997e-05 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489741+0000", 2024-09-21T23:10:06.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8144e-05 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.489755+0000", 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4270000000000001e-05 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491279+0000", 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001523834 2024-09-21T23:10:06.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491307+0000", 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8180000000000001e-05 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496464+0000", 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051571480000000003 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496499+0000", 2024-09-21T23:10:06.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5435999999999998e-05 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:138 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'96, mlcod=15'86)", 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.872781161999999, 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062358350000000003, 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.413 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490299+0000", 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490301+0000", 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.917e-06 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490315+0000", 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490344+0000", 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490361+0000", 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6883000000000001e-05 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492263+0000", 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001902405 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492296+0000", 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2650000000000001e-05 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496504+0000", 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042075949999999997 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496535+0000", 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1506e-05 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:139 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'97, mlcod=15'86)", 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.872164297000001, 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056577900000000002, 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:10:06.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490916+0000", 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490918+0000", 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3969999999999999e-06 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490929+0000", 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1443e-05 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490953+0000", 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4023e-05 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.490970+0000", 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6609999999999999e-05 2024-09-21T23:10:06.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492980+0000", 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020102779999999999 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493012+0000", 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1846e-05 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496539+0000", 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035270430000000001 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496574+0000", 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5150000000000001e-05 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:140 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'98, mlcod=15'86)", 2024-09-21T23:10:06.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.871543301999999, 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057984830000000001, 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491537+0000", 2024-09-21T23:10:06.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491539+0000", 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3629999999999999e-06 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491661+0000", 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012253899999999999 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491684+0000", 2024-09-21T23:10:06.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3173999999999999e-05 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.491701+0000", 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7105999999999999e-05 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493648+0000", 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001946309 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.493677+0000", 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9586000000000001e-05 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.497248+0000", 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035708100000000002 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.497336+0000", 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7596000000000006e-05 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4210.0:141 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'99, mlcod=15'86)", 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.870661419000001, 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005692557, 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492419+0000", 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492420+0000", 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492429+0000", 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0629999999999995e-06 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492446+0000", 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6307000000000002e-05 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.492457+0000", 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.154e-05 2024-09-21T23:10:06.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496632+0000", 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041749910000000003 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.496661+0000", 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9084000000000001e-05 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.498022+0000", 2024-09-21T23:10:06.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013602589999999999 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:42.498112+0000", 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0053e-05 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:10:06.431 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9832410740000004, 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063068619999999999, 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:10:06.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379839+0000", 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379842+0000", 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1780000000000002e-06 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379856+0000", 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4936999999999999e-05 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379878+0000", 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1248999999999999e-05 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379888+0000", 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0302999999999999e-05 2024-09-21T23:10:06.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379983+0000", 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4987000000000004e-05 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.379992+0000", 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.568e-06 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383095+0000", 2024-09-21T23:10:06.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031032379999999999 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383305+0000", 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020978199999999999 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.383621+0000", 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000316562 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.385245+0000", 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016241459999999999 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.385925+0000", 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067988399999999996 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.386038+0000", 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113081 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:57.386146+0000", 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107947 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:931 3.17 3:e906d0d7:::rbd_object_map.1149b00fdffd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.945994+0000", 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.41708642, 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046177170000000003, 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 931 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.945994+0000", 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.945994+0000", 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.945995+0000", 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7699999999999992e-07 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.946000+0000", 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6249999999999998e-06 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.946011+0000", 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1235e-05 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.946016+0000", 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.614e-06 2024-09-21T23:10:06.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.946070+0000", 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3060999999999999e-05 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.946160+0000", 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0198000000000006e-05 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.946578+0000", 2024-09-21T23:10:06.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000418556 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.949269+0000", 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0026911050000000001 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.950510+0000", 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012400989999999999 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.950557+0000", 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7621999999999999e-05 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.950612+0000", 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4625e-05 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:1036 3.17 3:e906d0d7:::rbd_object_map.1149b00fdffd:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.086427+0000", 2024-09-21T23:10:06.444 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.276653295, 2024-09-21T23:10:06.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004585174, 2024-09-21T23:10:06.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:06.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:06.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1036 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086427+0000", 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086427+0000", 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086429+0000", 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.289e-06 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086440+0000", 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0783e-05 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086463+0000", 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3238e-05 2024-09-21T23:10:06.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086475+0000", 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1849e-05 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086604+0000", 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129047 2024-09-21T23:10:06.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.086760+0000", 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000155236 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.087169+0000", 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040944799999999997 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.089319+0000", 2024-09-21T23:10:06.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021501459999999999 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.090834+0000", 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015145079999999999 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.090923+0000", 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9755999999999999e-05 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.091012+0000", 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8874000000000003e-05 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:1083 3.6 3:61ca63e2:::rbd_data.1149b00fdffd.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.192835+0000", 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.170245746, 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021384258, 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:06.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1083 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.192835+0000", 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.192835+0000", 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.192835+0000", 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8899999999999998e-07 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.192842+0000", 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3520000000000003e-06 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.192852+0000", 2024-09-21T23:10:06.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8670000000000006e-06 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.192857+0000", 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9559999999999996e-06 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.192976+0000", 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119398 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.193109+0000", 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013321200000000001 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.196990+0000", 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00388121 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.213975+0000", 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016984373000000001 2024-09-21T23:10:06.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.214062+0000", 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7171999999999998e-05 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.214144+0000", 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1676999999999997e-05 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.214219+0000", 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5252000000000004e-05 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop_reply(client.4425.0:1083 3.6 e27/21 ondisk, result = 0)", 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.203267+0000", 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1598129699999999, 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010971316, 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.203267+0000", 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.203267+0000", 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.203269+0000", 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-21T23:10:06.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.203273+0000", 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.941e-06 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.203284+0000", 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0919e-05 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.203290+0000", 2024-09-21T23:10:06.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2319999999999998e-06 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.214033+0000", 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010743345 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.214053+0000", 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9890000000000001e-05 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.214239+0000", 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000185799 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:1099 3.12 3:4e85fc25:::rbd_header.1149b00fdffd:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.404400+0000", 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.9586807039999998, 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048931460000000001, 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:06.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 1099 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404400+0000", 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404400+0000", 2024-09-21T23:10:06.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404402+0000", 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2010000000000002e-06 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404413+0000", 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1488000000000001e-05 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404431+0000", 2024-09-21T23:10:06.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8037e-05 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404444+0000", 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.259e-05 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404530+0000", 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5774999999999998e-05 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:06.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.404654+0000", 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000123736 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.405128+0000", 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00047476400000000002 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.407147+0000", 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020191139999999998 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:06.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.409162+0000", 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020147440000000002 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.409235+0000", 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3126000000000004e-05 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.409293+0000", 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7571000000000002e-05 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:06.467 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:06.467 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:06.468 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.468 INFO:teuthology.orchestra.run.smithi150.stdout:571 punch from 0x32eb3cf to 0x32ec4ee, (0x111f bytes) 2024-09-21T23:10:06.468 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.469 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:06.469 INFO:teuthology.orchestra.run.smithi150.stdout:575 writesame 0xc0f35a1 thru 0xc0f9d3c data_size 0x26(0x679c bytes) 2024-09-21T23:10:06.469 INFO:teuthology.orchestra.run.smithi150.stdout:576 write 0x583abde thru 0x583f31f (0x4742 bytes) 2024-09-21T23:10:06.469 INFO:teuthology.orchestra.run.smithi150.stdout:577 clone 5 order 23 su 32768 sc 14 2024-09-21T23:10:06.470 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T23:10:06.470 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:10:06.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T23:10:06.471 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:06.466+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:06.569 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:06.566+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:06.573 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone4 from 0xc0f9d3d (overlap 0x29d96a1) to 0x25aa12a 2024-09-21T23:10:06.644 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:06.644 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:06.645 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:06.645 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:06.662 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:06.703 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:06.702+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:06.736 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-21T23:10:06.770 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:06.770+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:06.826 INFO:teuthology.orchestra.run.smithi150.stdout:578 read 0xb825a90 thru 0xb82fff6 (0xa567 bytes) 2024-09-21T23:10:06.827 INFO:teuthology.orchestra.run.smithi150.stdout:579 punch from 0xbc6a892 to 0xbc6e0e6, (0x3854 bytes) 2024-09-21T23:10:06.834 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.835 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.840 INFO:teuthology.orchestra.run.smithi150.stdout:583 trunc from 0xc0f9d3d to 0x8900071 2024-09-21T23:10:06.850 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:06.850 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:06.850 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:06.850 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:06.850 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:06.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:06.871 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:06.870+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:06.875 INFO:teuthology.orchestra.run.smithi150.stdout:584 read 0x11407ea thru 0x114f45e (0xec75 bytes) 2024-09-21T23:10:06.877 INFO:teuthology.orchestra.run.smithi150.stdout:585 read 0x1484ad0 thru 0x148de28 (0x9359 bytes) 2024-09-21T23:10:06.879 INFO:teuthology.orchestra.run.smithi150.stdout:587 writesame 0x232630 thru 0x2415ff data_size 0x18(0xefd0 bytes) 2024-09-21T23:10:06.967 INFO:teuthology.orchestra.run.smithi150.stdout:588 writesame 0x75342e2 thru 0x753e4fa data_size 0x1(0xa219 bytes) 2024-09-21T23:10:06.972 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:06.970+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:06.995 INFO:teuthology.orchestra.run.smithi150.stdout:590 punch from 0x22829aa to 0x229040f, (0xda65 bytes) 2024-09-21T23:10:06.995 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:06.995 INFO:teuthology.orchestra.run.smithi150.stdout:593 read 0x6337982 thru 0x634464f (0xccce bytes) 2024-09-21T23:10:06.995 INFO:teuthology.orchestra.run.smithi150.stdout:595 write 0x115da41 thru 0x1160095 (0x2655 bytes) 2024-09-21T23:10:07.021 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:07.021 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3365 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2752512~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.881714+0000", 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13861305800000001, 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052020520000000004, 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.022 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3365 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881714+0000", 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881714+0000", 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881714+0000", 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881717+0000", 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.022e-06 2024-09-21T23:10:07.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881723+0000", 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8699999999999997e-06 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881730+0000", 2024-09-21T23:10:07.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5579999999999997e-06 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.886376+0000", 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046461280000000002 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.886507+0000", 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013105300000000001 2024-09-21T23:10:07.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.886916+0000", 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040890499999999999 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3366 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.028 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.881736+0000", 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13859142699999999, 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057305469999999999, 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3366 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881736+0000", 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881736+0000", 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881736+0000", 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881738+0000", 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4030000000000001e-06 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881742+0000", 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.659e-06 2024-09-21T23:10:07.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881745+0000", 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7889999999999999e-06 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.886955+0000", 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005210264 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.887046+0000", 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1243999999999996e-05 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.887466+0000", 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041994800000000001 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3367 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3670016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.881750+0000", 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.138577332, 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062483620000000004, 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.034 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3367 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881750+0000", 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881750+0000", 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881750+0000", 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3699999999999999e-07 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881751+0000", 2024-09-21T23:10:07.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881754+0000", 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3440000000000001e-06 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881757+0000", 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7240000000000001e-06 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.887523+0000", 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057661370000000002 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.887614+0000", 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1075000000000005e-05 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.887998+0000", 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038366499999999999 2024-09-21T23:10:07.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3368 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.881762+0000", 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.138564566, 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069604230000000003, 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3368 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881762+0000", 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881762+0000", 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881763+0000", 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.34e-07 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881764+0000", 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1510000000000001e-06 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881767+0000", 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4139999999999998e-06 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.881770+0000", 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7650000000000002e-06 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888062+0000", 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006291835 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888230+0000", 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016818200000000001 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888723+0000", 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049284199999999995 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3550 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2785280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.888413+0000", 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13191365399999999, 2024-09-21T23:10:07.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050483799999999999, 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3550 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888413+0000", 2024-09-21T23:10:07.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888413+0000", 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888414+0000", 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888416+0000", 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1789999999999998e-06 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888422+0000", 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3980000000000004e-06 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888427+0000", 2024-09-21T23:10:07.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7090000000000001e-06 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.892929+0000", 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045020930000000004 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.893014+0000", 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4965999999999999e-05 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.893462+0000", 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044761300000000003 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3551 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3244032~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.888434+0000", 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13189319999999999, 2024-09-21T23:10:07.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055690330000000001, 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3551 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888434+0000", 2024-09-21T23:10:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888434+0000", 2024-09-21T23:10:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888434+0000", 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888437+0000", 2024-09-21T23:10:07.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3810000000000002e-06 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888443+0000", 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7309999999999996e-06 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888448+0000", 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6070000000000001e-06 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.893492+0000", 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005043563 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.893562+0000", 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0205999999999998e-05 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894003+0000", 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044106000000000001 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3552 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3702784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.056 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.888455+0000", 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13187176, 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060053670000000002, 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3552 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888455+0000", 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888455+0000", 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888458+0000", 2024-09-21T23:10:07.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.728e-06 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888460+0000", 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.413e-06 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888467+0000", 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3860000000000004e-06 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888472+0000", 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0159999999999999e-06 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894034+0000", 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055621020000000002 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894100+0000", 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6487000000000002e-05 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894461+0000", 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036023499999999998 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3553 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 4161536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.888479+0000", 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13184827199999999, 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064895100000000004, 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.062 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3553 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888479+0000", 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888479+0000", 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888479+0000", 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.27e-07 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888481+0000", 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3420000000000002e-06 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888499+0000", 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7073999999999999e-05 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.888505+0000", 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5440000000000004e-06 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894492+0000", 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059874350000000002 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894591+0000", 2024-09-21T23:10:07.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8021000000000002e-05 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894968+0000", 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037766699999999999 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3728 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.067 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889025+0000", 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13130156800000001, 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063746360000000004, 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3728 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889025+0000", 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889025+0000", 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889026+0000", 2024-09-21T23:10:07.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889028+0000", 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9769999999999999e-06 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889034+0000", 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1419999999999998e-06 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889045+0000", 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1144999999999999e-05 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.894998+0000", 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005952906 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.895065+0000", 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7372999999999993e-05 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.895400+0000", 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033475699999999998 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.072 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3729 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889051+0000", 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.131275633, 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.006532281, 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3729 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889051+0000", 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889051+0000", 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889052+0000", 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6699999999999999e-07 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889055+0000", 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8870000000000001e-06 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889061+0000", 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0850000000000002e-06 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889065+0000", 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4009999999999999e-06 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.895430+0000", 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063647970000000002 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.895499+0000", 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9364999999999997e-05 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.895584+0000", 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4379e-05 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3730 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889072+0000", 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13125510100000001, 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069773420000000001, 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3730 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889072+0000", 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889072+0000", 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889072+0000", 2024-09-21T23:10:07.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0299999999999999e-07 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889074+0000", 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889081+0000", 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889086+0000", 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8910000000000003e-06 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.895612+0000", 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065264490000000001 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.895697+0000", 2024-09-21T23:10:07.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4202999999999993e-05 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896049+0000", 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035252200000000002 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.082 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3731 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889092+0000", 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.131234612, 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073843570000000002, 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3731 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.083 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889092+0000", 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889092+0000", 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889093+0000", 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0500000000000003e-07 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889095+0000", 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2680000000000001e-06 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889101+0000", 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9950000000000002e-06 2024-09-21T23:10:07.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889106+0000", 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.442e-06 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896079+0000", 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069736829999999996 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896150+0000", 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896477+0000", 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000326378 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3732 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889111+0000", 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13121575999999999, 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078260259999999998, 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3732 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889111+0000", 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889111+0000", 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889112+0000", 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889131+0000", 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9694000000000002e-05 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889138+0000", 2024-09-21T23:10:07.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0589999999999997e-06 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889149+0000", 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0217999999999999e-05 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896507+0000", 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073588400000000002 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896575+0000", 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7312999999999997e-05 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896937+0000", 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036243499999999998 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3733 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2359296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889156+0000", 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.131170495, 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082351119999999993, 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.093 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3733 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889156+0000", 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889156+0000", 2024-09-21T23:10:07.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889157+0000", 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1500000000000004e-07 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889160+0000", 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4490000000000002e-06 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889166+0000", 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4949999999999996e-06 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889171+0000", 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6689999999999997e-06 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896967+0000", 2024-09-21T23:10:07.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077958599999999999 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.897035+0000", 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8232999999999995e-05 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.897392+0000", 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035679100000000002 2024-09-21T23:10:07.097 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3734 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889177+0000", 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13115032700000001, 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097532060000000004, 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3734 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889177+0000", 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889177+0000", 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889177+0000", 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4999999999999998e-07 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889179+0000", 2024-09-21T23:10:07.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9989999999999998e-06 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889185+0000", 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7899999999999996e-06 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889189+0000", 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.897422+0000", 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082330930000000004 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.897488+0000", 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5864999999999994e-05 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.898930+0000", 2024-09-21T23:10:07.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014419560000000001 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3735 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889202+0000", 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13112475500000001, 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011023029, 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.103 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3735 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889202+0000", 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889202+0000", 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889203+0000", 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889222+0000", 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9182999999999999e-05 2024-09-21T23:10:07.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889230+0000", 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7919999999999993e-06 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889235+0000", 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.898990+0000", 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097550870000000008 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.899655+0000", 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066406699999999996 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.900225+0000", 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00057069499999999997 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3736 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.889245+0000", 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.13108188200000001, 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011551926000000001, 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3736 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889245+0000", 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889245+0000", 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889245+0000", 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3700000000000001e-07 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889248+0000", 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889254+0000", 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2759999999999997e-06 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.889259+0000", 2024-09-21T23:10:07.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5059999999999997e-06 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.900274+0000", 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011015252999999999 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.900407+0000", 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013347500000000001 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.900797+0000", 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038955199999999999 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3835 3.4 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:7 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.891380+0000", 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.1289467, 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051473960000000003, 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3835 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891380+0000", 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891380+0000", 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891381+0000", 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891390+0000", 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0720000000000006e-06 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891394+0000", 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7350000000000004e-06 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891398+0000", 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-21T23:10:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896386+0000", 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049880250000000001 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896462+0000", 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6179000000000004e-05 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896528+0000", 2024-09-21T23:10:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5889000000000006e-05 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3836 3.4 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:7 [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.891403+0000", 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.12892347800000001, 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052978019999999999, 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3836 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891403+0000", 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891403+0000", 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891404+0000", 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6e-07 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891405+0000", 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4839999999999999e-06 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891409+0000", 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4510000000000001e-06 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891411+0000", 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5730000000000002e-06 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896555+0000", 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051440449999999999 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896628+0000", 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2653999999999999e-05 2024-09-21T23:10:07.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.896701+0000", 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3335000000000002e-05 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:3837 3.4 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:7 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.891415+0000", 2024-09-21T23:10:07.123 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.128912316, 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0057780069999999999, 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 3837 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891415+0000", 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891415+0000", 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891415+0000", 2024-09-21T23:10:07.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4400000000000001e-07 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891416+0000", 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.246e-06 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891419+0000", 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.891422+0000", 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7640000000000001e-06 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.897018+0000", 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0055961880000000002 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.897098+0000", 2024-09-21T23:10:07.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9629000000000006e-05 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.897193+0000", 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.48e-05 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.128 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:07.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:07.131 INFO:teuthology.orchestra.run.smithi150.stdout:596 trunc from 0x8900071 to 0xa17dd36 2024-09-21T23:10:07.131 INFO:teuthology.orchestra.run.smithi150.stdout:598 writesame 0x107b441 thru 0x107b522 data_size 0x1(0xe2 bytes) 2024-09-21T23:10:07.131 INFO:teuthology.orchestra.run.smithi150.stdout:599 trunc from 0xa17dd36 to 0x7544a15 2024-09-21T23:10:07.131 INFO:teuthology.orchestra.run.smithi150.stdout:600 writesame 0x9ca6680 thru 0x9ca7954 data_size 0x1(0x12d5 bytes) 2024-09-21T23:10:07.131 INFO:teuthology.orchestra.run.smithi150.stdout:601 read 0x9457c7 thru 0x94efdf (0x9819 bytes) 2024-09-21T23:10:07.131 INFO:teuthology.orchestra.run.smithi150.stdout:603 write 0xb4f57a0 thru 0xb4fad4e (0x55af bytes) 2024-09-21T23:10:07.132 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:07.130+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:07.134 INFO:teuthology.orchestra.run.smithi150.stdout:606 read 0x19a1b47 thru 0x19a35ac (0x1a66 bytes) 2024-09-21T23:10:07.150 INFO:teuthology.orchestra.run.smithi150.stdout:607 trunc from 0xb4fad4f to 0x23a980a 2024-09-21T23:10:07.231 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:07.230+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:07.287 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:07.287 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:07.303 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:07.332 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:07.330+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:07.338 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:07.338 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:07.338 INFO:teuthology.orchestra.run.smithi150.stdout:611 clone 6 order 20 su 16384 sc 12 2024-09-21T23:10:07.433 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:07.434+0000 7f48b0e1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26594) UID: 0 2024-09-21T23:10:07.436 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T23:10:07.454 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:07.455 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:07.462 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:10:07.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:10:07.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:07.535 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:07.534+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:07.580 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone5 from 0x23a980a (overlap 0x23a980a) to 0x721e68 2024-09-21T23:10:07.636 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:07.634+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:07.641 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:10:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 21.925492971000001, 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039594890000000001, 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716126+0000", 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716131+0000", 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716144+0000", 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2267e-05 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716158+0000", 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4731e-05 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716239+0000", 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1013999999999994e-05 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716271+0000", 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1535999999999998e-05 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720016+0000", 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037447399999999999 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720084+0000", 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8813999999999995e-05 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:550 3.4 e25/21 3:251a9a20:::rbd_data.11499f70651.0000000000000005:head v 25'14, mlcod=25'12)", 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:01.361381+0000", 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.2802373209999995, 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00094816900000000001, 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361381+0000", 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361381+0000", 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361382+0000", 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.429e-06 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361401+0000", 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8924999999999999e-05 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361416+0000", 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5531000000000001e-05 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361424+0000", 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0620000000000003e-06 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361506+0000", 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2894999999999998e-05 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.361534+0000", 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7949999999999998e-05 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362291+0000", 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00075628999999999996 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:01.362329+0000", 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8087000000000003e-05 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:725 3.0 e27/21 3:04479396:::rbd_id.image_client.0-clone2:head v 27'2, mlcod=0'0)", 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1697609120000001, 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010753119999999999, 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471857+0000", 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7e-07 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471866+0000", 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6570000000000003e-06 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471881+0000", 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4684e-05 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471891+0000", 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0424999999999999e-05 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471959+0000", 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8324000000000005e-05 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.471985+0000", 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5252000000000001e-05 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.472877+0000", 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00089217400000000003 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.472932+0000", 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5426000000000001e-05 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:933 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'1, mlcod=0'0)", 2024-09-21T23:10:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6895027310000001, 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001140825, 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952116+0000", 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952303+0000", 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187238 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952317+0000", 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4348000000000001e-05 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952326+0000", 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5439999999999999e-06 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952386+0000", 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0309000000000001e-05 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952405+0000", 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9208999999999999e-05 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.953214+0000", 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00080896899999999999 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.953256+0000", 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1714000000000001e-05 2024-09-21T23:10:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:945 3.9 e27/21 3:96c84604:::rbd_data.1149b00fdffd.0000000000000006:head v 27'2, mlcod=0'0)", 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6679568649999998, 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001850607, 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973663+0000", 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.728e-06 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973731+0000", 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7798000000000004e-05 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973749+0000", 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8703e-05 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973759+0000", 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7890000000000006e-06 2024-09-21T23:10:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973897+0000", 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138422 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973941+0000", 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3192999999999998e-05 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.975430+0000", 2024-09-21T23:10:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001489499 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.975512+0000", 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1475000000000002e-05 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:947 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'3, mlcod=27'1)", 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6637029089999995, 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014565820000000001, 2024-09-21T23:10:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977916+0000", 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977965+0000", 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8903000000000003e-05 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977979+0000", 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4579999999999999e-05 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977987+0000", 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3579999999999997e-06 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.978125+0000", 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013831600000000001 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.978170+0000", 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5346000000000003e-05 2024-09-21T23:10:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.979284+0000", 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011134579999999999 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.979372+0000", 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7773000000000001e-05 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:949 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'4, mlcod=27'2)", 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6446241439999998, 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012844040000000001, 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996995+0000", 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3480000000000001e-06 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997005+0000", 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4879999999999996e-06 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997036+0000", 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1080000000000001e-05 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997046+0000", 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0338e-05 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997191+0000", 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000145069 2024-09-21T23:10:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997231+0000", 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0309000000000003e-05 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998169+0000", 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00093785299999999999 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998278+0000", 2024-09-21T23:10:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010891900000000001 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:951 3.9 e27/21 3:96c84604:::rbd_data.1149b00fdffd.0000000000000006:head v 27'5, mlcod=27'2)", 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6435461279999997, 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011579310000000001, 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998073+0000", 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6929999999999999e-06 2024-09-21T23:10:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998107+0000", 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.307e-05 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998132+0000", 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5420000000000001e-05 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998146+0000", 2024-09-21T23:10:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4463e-05 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998324+0000", 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017724200000000001 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998346+0000", 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2334e-05 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.999157+0000", 2024-09-21T23:10:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081133300000000002 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.999230+0000", 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2376e-05 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:1072 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'6, mlcod=27'4)", 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.166638+0000", 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.4749799870000002, 2024-09-21T23:10:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00096370500000000003, 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166638+0000", 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166638+0000", 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166640+0000", 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.688e-06 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166667+0000", 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7892e-05 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166690+0000", 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2459999999999998e-05 2024-09-21T23:10:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166700+0000", 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0044000000000001e-05 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166763+0000", 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3058000000000005e-05 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.166786+0000", 2024-09-21T23:10:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3436999999999999e-05 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.167565+0000", 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00077828499999999998 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.167602+0000", 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6841000000000002e-05 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:1073 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'7, mlcod=27'5)", 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.4595555649999996, 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001363263, 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182065+0000", 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4569999999999999e-06 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182082+0000", 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6736e-05 2024-09-21T23:10:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182113+0000", 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1412999999999999e-05 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182129+0000", 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5996999999999999e-05 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182268+0000", 2024-09-21T23:10:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013877900000000001 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182311+0000", 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2942999999999999e-05 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.183338+0000", 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010276619999999999 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.183426+0000", 2024-09-21T23:10:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7275999999999996e-05 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2850 3.4 e30/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 30'30, mlcod=30'28)", 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.6156015479999999, 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033350860000000001, 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026017+0000", 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.68e-07 2024-09-21T23:10:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026023+0000", 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1959999999999996e-06 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026033+0000", 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8120000000000002e-06 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026042+0000", 2024-09-21T23:10:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7059999999999993e-06 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026127+0000", 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4973000000000001e-05 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026145+0000", 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8422000000000001e-05 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029303+0000", 2024-09-21T23:10:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031578980000000001 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029351+0000", 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8411000000000001e-05 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2900 3.4 e30/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 30'31, mlcod=30'29)", 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.136827+0000", 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.504791062, 2024-09-21T23:10:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001321544, 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136827+0000", 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136827+0000", 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136828+0000", 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136847+0000", 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9488e-05 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136863+0000", 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5658000000000002e-05 2024-09-21T23:10:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136872+0000", 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7269999999999996e-06 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136999+0000", 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012732299999999999 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.137023+0000", 2024-09-21T23:10:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3574000000000002e-05 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138097+0000", 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001074158 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138148+0000", 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1666000000000003e-05 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2902 3.0 e30/21 3:02fe727d:::rbd_data.114912f5d0f.000000000000000d:head v 30'11, mlcod=30'9)", 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5047231650000001, 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012365950000000001, 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136900+0000", 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7509999999999999e-06 2024-09-21T23:10:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136909+0000", 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9129999999999992e-06 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136914+0000", 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5820000000000001e-06 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.137182+0000", 2024-09-21T23:10:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026748899999999999 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.137197+0000", 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5447999999999998e-05 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138071+0000", 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087370000000000004 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138131+0000", 2024-09-21T23:10:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0297000000000002e-05 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:3152 3.4 e31/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 31'33, mlcod=30'30)", 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0360219909999999, 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018085410000000001, 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3700000000000001e-07 2024-09-21T23:10:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605604+0000", 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6669999999999996e-06 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605627+0000", 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3167e-05 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605638+0000", 2024-09-21T23:10:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0959e-05 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605704+0000", 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5925000000000003e-05 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605728+0000", 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3765e-05 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.607361+0000", 2024-09-21T23:10:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016327340000000001 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.607404+0000", 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3887000000000002e-05 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:3154 3.0 e31/21 3:02fe727d:::rbd_data.114912f5d0f.000000000000000d:head v 31'14, mlcod=30'11)", 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.608529+0000", 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.033089267, 2024-09-21T23:10:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013675969999999999, 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608529+0000", 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608529+0000", 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608530+0000", 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608537+0000", 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4959999999999999e-06 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608552+0000", 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5304000000000001e-05 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608560+0000", 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0620000000000001e-06 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608624+0000", 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3732000000000006e-05 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608650+0000", 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5842000000000001e-05 2024-09-21T23:10:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.609847+0000", 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001196768 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.609896+0000", 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9496000000000002e-05 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:3330 3.0 e31/21 3:0763e436:::rbd_data.1149d3a069d9.000000000000000f:head v 31'15, mlcod=30'12)", 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.856265+0000", 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.78535291100000004, 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010899320000000001, 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856265+0000", 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856265+0000", 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856268+0000", 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856286+0000", 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8029999999999998e-05 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856331+0000", 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4982999999999999e-05 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856353+0000", 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1596000000000001e-05 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856452+0000", 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9378999999999998e-05 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.856474+0000", 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.857271+0000", 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079684600000000003 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.857355+0000", 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3812000000000003e-05 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:4287 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'16, mlcod=31'14)", 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.61224953599999998, 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015395872, 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029371+0000", 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2759999999999999e-06 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031303+0000", 2024-09-21T23:10:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019322359999999999 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031331+0000", 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8408999999999999e-05 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031342+0000", 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1086e-05 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031428+0000", 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5990999999999997e-05 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031454+0000", 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044695+0000", 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013240784 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044764+0000", 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9623999999999996e-05 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:4299 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'17, mlcod=31'15)", 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.070721+0000", 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.57089639999999997, 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014312190000000001, 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070721+0000", 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070721+0000", 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070724+0000", 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.83e-06 2024-09-21T23:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070742+0000", 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070782+0000", 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9752999999999997e-05 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070798+0000", 2024-09-21T23:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6816999999999998e-05 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070878+0000", 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9777000000000005e-05 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070901+0000", 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3006999999999998e-05 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.072107+0000", 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012055939999999999 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.072153+0000", 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5998000000000002e-05 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:5384 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'20, mlcod=31'18)", 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.201914+0000", 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.43970386500000003, 2024-09-21T23:10:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021100590000000001, 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201914+0000", 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201914+0000", 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201915+0000", 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3999999999999999e-06 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201925+0000", 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0002e-05 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201948+0000", 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2586e-05 2024-09-21T23:10:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201958+0000", 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0076e-05 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.202048+0000", 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0012999999999994e-05 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.202093+0000", 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4799000000000003e-05 2024-09-21T23:10:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.203959+0000", 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001865853 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.204024+0000", 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.533e-05 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6012 3.4 e31/21 3:21794b57:::rbd_data.1149d3a069d9.0000000000000003:head v 31'34, mlcod=30'31)", 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.37322630000000001, 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094147550000000003, 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268393+0000", 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.72e-06 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269210+0000", 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081656700000000003 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269235+0000", 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5000000000000001e-05 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269248+0000", 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3393e-05 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269314+0000", 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6180000000000007e-05 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269341+0000", 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6554999999999999e-05 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277723+0000", 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083823960000000003 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277806+0000", 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2943999999999997e-05 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:07.728 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:07.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:07.736 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:07.734+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:07.736 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-21T23:10:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:10:07.774 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:10:07.791 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-21T23:10:07.441784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":283094701,"num_objects":90,"num_object_clones":16,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":21,"num_read":4483,"num_read_kb":5764,"num_write":875,"num_write_kb":16146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":727,"ondisk_log_size":727,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231652,"kb_used_data":11216,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754743068,"statfs":{"total":773094113280,"available":772856901632,"internally_reserved":0,"allocated":11485184,"data_stored":9591580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":280986635,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":21,"num_read":3981,"num_read_kb":5346,"num_write":667,"num_write_kb":13556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039586"},"pg_stats":[{"pgid":"3.1f","version":"24'1","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381528+0000","last_change":"2024-09-21T23:09:57.384622+0000","last_active":"2024-09-21T23:10:05.381528+0000","last_peered":"2024-09-21T23:10:05.381528+0000","last_clean":"2024-09-21T23:10:05.381528+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:10:05.381528+0000","last_undegraded":"2024-09-21T23:10:05.381528+0000","last_fullsized":"2024-09-21T23:10:05.381528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:12.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018862200000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362348+0000","last_change":"2024-09-21T23:09:57.368121+0000","last_active":"2024-09-21T23:10:05.362348+0000","last_peered":"2024-09-21T23:10:05.362348+0000","last_clean":"2024-09-21T23:10:05.362348+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:10:05.362348+0000","last_undegraded":"2024-09-21T23:10:05.362348+0000","last_fullsized":"2024-09-21T23:10:05.362348+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:20:19.079775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010293570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"30'24","reported_seq":110,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420340+0000","last_change":"2024-09-21T23:09:57.379419+0000","last_active":"2024-09-21T23:10:05.420340+0000","last_peered":"2024-09-21T23:10:05.420340+0000","last_clean":"2024-09-21T23:10:05.420340+0000","last_became_active":"2024-09-21T23:09:55.383016+0000","last_became_peered":"2024-09-21T23:09:55.383016+0000","last_unstale":"2024-09-21T23:10:05.420340+0000","last_undegraded":"2024-09-21T23:10:05.420340+0000","last_fullsized":"2024-09-21T23:10:05.420340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:31:19.672645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057415099999999996,"stat_sum":{"num_bytes":25690112,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":62,"num_read_kb":229,"num_write":22,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"29'16","reported_seq":69,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420235+0000","last_change":"2024-09-21T23:09:57.380029+0000","last_active":"2024-09-21T23:10:05.420235+0000","last_peered":"2024-09-21T23:10:05.420235+0000","last_clean":"2024-09-21T23:10:05.420235+0000","last_became_active":"2024-09-21T23:09:55.385836+0000","last_became_peered":"2024-09-21T23:09:55.385836+0000","last_unstale":"2024-09-21T23:10:05.420235+0000","last_undegraded":"2024-09-21T23:10:05.420235+0000","last_fullsized":"2024-09-21T23:10:05.420235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:03.351360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090318899999999999,"stat_sum":{"num_bytes":811057,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":126,"num_write":14,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"29'2","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381457+0000","last_change":"2024-09-21T23:09:57.383801+0000","last_active":"2024-09-21T23:10:05.381457+0000","last_peered":"2024-09-21T23:10:05.381457+0000","last_clean":"2024-09-21T23:10:05.381457+0000","last_became_active":"2024-09-21T23:09:55.377005+0000","last_became_peered":"2024-09-21T23:09:55.377005+0000","last_unstale":"2024-09-21T23:10:05.381457+0000","last_undegraded":"2024-09-21T23:10:05.381457+0000","last_fullsized":"2024-09-21T23:10:05.381457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:14:18.281291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007694,"stat_sum":{"num_bytes":61,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"30'85","reported_seq":233,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460701+0000","last_change":"2024-09-21T23:09:57.369613+0000","last_active":"2024-09-21T23:10:05.460701+0000","last_peered":"2024-09-21T23:10:05.460701+0000","last_clean":"2024-09-21T23:10:05.460701+0000","last_became_active":"2024-09-21T23:09:55.378756+0000","last_became_peered":"2024-09-21T23:09:55.378756+0000","last_unstale":"2024-09-21T23:10:05.460701+0000","last_undegraded":"2024-09-21T23:10:05.460701+0000","last_fullsized":"2024-09-21T23:10:05.460701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:59.643930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011201469999999999,"stat_sum":{"num_bytes":49,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":480,"num_read_kb":399,"num_write":129,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.459911+0000","last_change":"2024-09-21T23:09:57.374916+0000","last_active":"2024-09-21T23:10:05.459911+0000","last_peered":"2024-09-21T23:10:05.459911+0000","last_clean":"2024-09-21T23:10:05.459911+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:10:05.459911+0000","last_undegraded":"2024-09-21T23:10:05.459911+0000","last_fullsized":"2024-09-21T23:10:05.459911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:42.428614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073844600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"29'5","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361704+0000","last_change":"2024-09-21T23:09:57.367436+0000","last_active":"2024-09-21T23:10:05.361704+0000","last_peered":"2024-09-21T23:10:05.361704+0000","last_clean":"2024-09-21T23:10:05.361704+0000","last_became_active":"2024-09-21T23:09:55.366388+0000","last_became_peered":"2024-09-21T23:09:55.366388+0000","last_unstale":"2024-09-21T23:10:05.361704+0000","last_undegraded":"2024-09-21T23:10:05.361704+0000","last_fullsized":"2024-09-21T23:10:05.361704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:28.195405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054973999999999995,"stat_sum":{"num_bytes":2339902,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"30'48","reported_seq":119,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.506544+0000","last_change":"2024-09-21T23:09:57.384681+0000","last_active":"2024-09-21T23:10:05.506544+0000","last_peered":"2024-09-21T23:10:05.506544+0000","last_clean":"2024-09-21T23:10:05.506544+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:10:05.506544+0000","last_undegraded":"2024-09-21T23:10:05.506544+0000","last_fullsized":"2024-09-21T23:10:05.506544+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:46:55.092696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001819836,"stat_sum":{"num_bytes":27011594,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":147,"num_read_kb":328,"num_write":53,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"29'3","reported_seq":281,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.519118+0000","last_change":"2024-09-21T23:09:57.379117+0000","last_active":"2024-09-21T23:10:05.519118+0000","last_peered":"2024-09-21T23:10:05.519118+0000","last_clean":"2024-09-21T23:10:05.519118+0000","last_became_active":"2024-09-21T23:09:55.388499+0000","last_became_peered":"2024-09-21T23:09:55.388499+0000","last_unstale":"2024-09-21T23:10:05.519118+0000","last_undegraded":"2024-09-21T23:10:05.519118+0000","last_fullsized":"2024-09-21T23:10:05.519118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:11:30.098231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049884500000000004,"stat_sum":{"num_bytes":26451551,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":246,"num_read_kb":416,"num_write":5,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'82","reported_seq":529,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.437082+0000","last_change":"2024-09-21T23:09:57.392268+0000","last_active":"2024-09-21T23:10:05.437082+0000","last_peered":"2024-09-21T23:10:05.437082+0000","last_clean":"2024-09-21T23:10:05.437082+0000","last_became_active":"2024-09-21T23:09:55.384020+0000","last_became_peered":"2024-09-21T23:09:55.384020+0000","last_unstale":"2024-09-21T23:10:05.437082+0000","last_undegraded":"2024-09-21T23:10:05.437082+0000","last_fullsized":"2024-09-21T23:10:05.437082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:56.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313919,"stat_sum":{"num_bytes":25024047,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":1181,"num_read_kb":1030,"num_write":73,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"30'20","reported_seq":69,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.422446+0000","last_change":"2024-09-21T23:09:57.369003+0000","last_active":"2024-09-21T23:10:05.422446+0000","last_peered":"2024-09-21T23:10:05.422446+0000","last_clean":"2024-09-21T23:10:05.422446+0000","last_became_active":"2024-09-21T23:09:55.389660+0000","last_became_peered":"2024-09-21T23:09:55.389660+0000","last_unstale":"2024-09-21T23:10:05.422446+0000","last_undegraded":"2024-09-21T23:10:05.422446+0000","last_fullsized":"2024-09-21T23:10:05.422446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:52.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110800000000001,"stat_sum":{"num_bytes":3384830,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":25,"num_read_kb":50,"num_write":16,"num_write_kb":359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.374419+0000","last_change":"2024-09-21T23:09:48.068366+0000","last_active":"2024-09-21T23:10:05.374419+0000","last_peered":"2024-09-21T23:10:05.374419+0000","last_clean":"2024-09-21T23:10:05.374419+0000","last_became_active":"2024-09-21T23:09:45.718621+0000","last_became_peered":"2024-09-21T23:09:45.718621+0000","last_unstale":"2024-09-21T23:10:05.374419+0000","last_undegraded":"2024-09-21T23:10:05.374419+0000","last_fullsized":"2024-09-21T23:10:05.374419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:44:48.782244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'6","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381715+0000","last_change":"2024-09-21T23:09:57.383456+0000","last_active":"2024-09-21T23:10:05.381715+0000","last_peered":"2024-09-21T23:10:05.381715+0000","last_clean":"2024-09-21T23:10:05.381715+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:10:05.381715+0000","last_undegraded":"2024-09-21T23:10:05.381715+0000","last_fullsized":"2024-09-21T23:10:05.381715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:19:39.549662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069300299999999998,"stat_sum":{"num_bytes":32635978,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":83,"num_write":5,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.459944+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:10:05.459944+0000","last_peered":"2024-09-21T23:10:05.459944+0000","last_clean":"2024-09-21T23:10:05.459944+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:10:05.459944+0000","last_undegraded":"2024-09-21T23:10:05.459944+0000","last_fullsized":"2024-09-21T23:10:05.459944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"29'18","reported_seq":82,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362657+0000","last_change":"2024-09-21T23:09:57.368949+0000","last_active":"2024-09-21T23:10:05.362657+0000","last_peered":"2024-09-21T23:10:05.362657+0000","last_clean":"2024-09-21T23:10:05.362657+0000","last_became_active":"2024-09-21T23:09:55.382399+0000","last_became_peered":"2024-09-21T23:09:55.382399+0000","last_unstale":"2024-09-21T23:10:05.362657+0000","last_undegraded":"2024-09-21T23:10:05.362657+0000","last_fullsized":"2024-09-21T23:10:05.362657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:10:33.832274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333359999999999,"stat_sum":{"num_bytes":25482042,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":45,"num_read_kb":460,"num_write":15,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.367194+0000","last_change":"2024-09-21T23:09:48.068555+0000","last_active":"2024-09-21T23:10:05.367194+0000","last_peered":"2024-09-21T23:10:05.367194+0000","last_clean":"2024-09-21T23:10:05.367194+0000","last_became_active":"2024-09-21T23:09:45.869065+0000","last_became_peered":"2024-09-21T23:09:45.869065+0000","last_unstale":"2024-09-21T23:10:05.367194+0000","last_undegraded":"2024-09-21T23:10:05.367194+0000","last_fullsized":"2024-09-21T23:10:05.367194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:48.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088130999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"30'18","reported_seq":59,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460071+0000","last_change":"2024-09-21T23:09:57.375144+0000","last_active":"2024-09-21T23:10:05.460071+0000","last_peered":"2024-09-21T23:10:05.460071+0000","last_clean":"2024-09-21T23:10:05.460071+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:10:05.460071+0000","last_undegraded":"2024-09-21T23:10:05.460071+0000","last_fullsized":"2024-09-21T23:10:05.460071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:41.785855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079926499999999996,"stat_sum":{"num_bytes":4521984,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":85,"num_write":18,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460213+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:10:05.460213+0000","last_peered":"2024-09-21T23:10:05.460213+0000","last_clean":"2024-09-21T23:10:05.460213+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:10:05.460213+0000","last_undegraded":"2024-09-21T23:10:05.460213+0000","last_fullsized":"2024-09-21T23:10:05.460213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"29'6","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420367+0000","last_change":"2024-09-21T23:09:57.378889+0000","last_active":"2024-09-21T23:10:05.420367+0000","last_peered":"2024-09-21T23:10:05.420367+0000","last_clean":"2024-09-21T23:10:05.420367+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:10:05.420367+0000","last_undegraded":"2024-09-21T23:10:05.420367+0000","last_fullsized":"2024-09-21T23:10:05.420367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:45:17.033071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000364898,"stat_sum":{"num_bytes":248866,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":9,"num_read_kb":2,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'12","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460183+0000","last_change":"2024-09-21T23:09:57.368994+0000","last_active":"2024-09-21T23:10:05.460183+0000","last_peered":"2024-09-21T23:10:05.460183+0000","last_clean":"2024-09-21T23:10:05.460183+0000","last_became_active":"2024-09-21T23:09:55.377435+0000","last_became_peered":"2024-09-21T23:09:55.377435+0000","last_unstale":"2024-09-21T23:10:05.460183+0000","last_undegraded":"2024-09-21T23:10:05.460183+0000","last_fullsized":"2024-09-21T23:10:05.460183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:18:14.697224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060347799999999996,"stat_sum":{"num_bytes":5570560,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":12,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420742+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:10:05.420742+0000","last_peered":"2024-09-21T23:10:05.420742+0000","last_clean":"2024-09-21T23:10:05.420742+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:10:05.420742+0000","last_undegraded":"2024-09-21T23:10:05.420742+0000","last_fullsized":"2024-09-21T23:10:05.420742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"29'4","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460007+0000","last_change":"2024-09-21T23:09:57.374883+0000","last_active":"2024-09-21T23:10:05.460007+0000","last_peered":"2024-09-21T23:10:05.460007+0000","last_clean":"2024-09-21T23:10:05.460007+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:10:05.460007+0000","last_undegraded":"2024-09-21T23:10:05.460007+0000","last_fullsized":"2024-09-21T23:10:05.460007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:34.370102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060209499999999997,"stat_sum":{"num_bytes":4543637,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":109,"num_write":4,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361290+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:10:05.361290+0000","last_peered":"2024-09-21T23:10:05.361290+0000","last_clean":"2024-09-21T23:10:05.361290+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:10:05.361290+0000","last_undegraded":"2024-09-21T23:10:05.361290+0000","last_fullsized":"2024-09-21T23:10:05.361290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"29'3","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381224+0000","last_change":"2024-09-21T23:09:57.385279+0000","last_active":"2024-09-21T23:10:05.381224+0000","last_peered":"2024-09-21T23:10:05.381224+0000","last_clean":"2024-09-21T23:10:05.381224+0000","last_became_active":"2024-09-21T23:09:55.376738+0000","last_became_peered":"2024-09-21T23:09:55.376738+0000","last_unstale":"2024-09-21T23:10:05.381224+0000","last_undegraded":"2024-09-21T23:10:05.381224+0000","last_fullsized":"2024-09-21T23:10:05.381224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:10:35.904378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021610209999999999,"stat_sum":{"num_bytes":45,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":5,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.365860+0000","last_change":"2024-09-21T23:09:48.068542+0000","last_active":"2024-09-21T23:10:05.365860+0000","last_peered":"2024-09-21T23:10:05.365860+0000","last_clean":"2024-09-21T23:10:05.365860+0000","last_became_active":"2024-09-21T23:09:45.714516+0000","last_became_peered":"2024-09-21T23:09:45.714516+0000","last_unstale":"2024-09-21T23:10:05.365860+0000","last_undegraded":"2024-09-21T23:10:05.365860+0000","last_fullsized":"2024-09-21T23:10:05.365860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:51.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000999572,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"29'18","reported_seq":61,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362145+0000","last_change":"2024-09-21T23:09:57.368695+0000","last_active":"2024-09-21T23:10:05.362145+0000","last_peered":"2024-09-21T23:10:05.362145+0000","last_clean":"2024-09-21T23:10:05.362145+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:10:05.362145+0000","last_undegraded":"2024-09-21T23:10:05.362145+0000","last_fullsized":"2024-09-21T23:10:05.362145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:40.264877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013976220000000001,"stat_sum":{"num_bytes":2293820,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":45,"num_write":26,"num_write_kb":2035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.419992+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:10:05.419992+0000","last_peered":"2024-09-21T23:10:05.419992+0000","last_clean":"2024-09-21T23:10:05.419992+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:10:05.419992+0000","last_undegraded":"2024-09-21T23:10:05.419992+0000","last_fullsized":"2024-09-21T23:10:05.419992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.366044+0000","last_change":"2024-09-21T23:09:41.675036+0000","last_active":"2024-09-21T23:10:05.366044+0000","last_peered":"2024-09-21T23:10:05.366044+0000","last_clean":"2024-09-21T23:10:05.366044+0000","last_became_active":"2024-09-21T23:09:41.674018+0000","last_became_peered":"2024-09-21T23:09:41.674018+0000","last_unstale":"2024-09-21T23:10:05.366044+0000","last_undegraded":"2024-09-21T23:10:05.366044+0000","last_fullsized":"2024-09-21T23:10:05.366044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:10.788395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"29'1","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361619+0000","last_change":"2024-09-21T23:09:57.367811+0000","last_active":"2024-09-21T23:10:05.361619+0000","last_peered":"2024-09-21T23:10:05.361619+0000","last_clean":"2024-09-21T23:10:05.361619+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:10:05.361619+0000","last_undegraded":"2024-09-21T23:10:05.361619+0000","last_fullsized":"2024-09-21T23:10:05.361619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:49:38.489900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094978,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"29'10","reported_seq":61,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362724+0000","last_change":"2024-09-21T23:09:57.369343+0000","last_active":"2024-09-21T23:10:05.362724+0000","last_peered":"2024-09-21T23:10:05.362724+0000","last_clean":"2024-09-21T23:10:05.362724+0000","last_became_active":"2024-09-21T23:09:55.767547+0000","last_became_peered":"2024-09-21T23:09:55.767547+0000","last_unstale":"2024-09-21T23:10:05.362724+0000","last_undegraded":"2024-09-21T23:10:05.362724+0000","last_fullsized":"2024-09-21T23:10:05.362724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:20.771000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084957200000000004,"stat_sum":{"num_bytes":2342126,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":19,"num_read_kb":34,"num_write":10,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"29'12","reported_seq":83,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.374433+0000","last_change":"2024-09-21T23:09:57.494957+0000","last_active":"2024-09-21T23:10:05.374433+0000","last_peered":"2024-09-21T23:10:05.374433+0000","last_clean":"2024-09-21T23:10:05.374433+0000","last_became_active":"2024-09-21T23:09:55.386143+0000","last_became_peered":"2024-09-21T23:09:55.386143+0000","last_unstale":"2024-09-21T23:10:05.374433+0000","last_undegraded":"2024-09-21T23:10:05.374433+0000","last_fullsized":"2024-09-21T23:10:05.374433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:53.517804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073170699999999995,"stat_sum":{"num_bytes":1540310,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":51,"num_read_kb":61,"num_write":10,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"29'8","reported_seq":46,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361810+0000","last_change":"2024-09-21T23:09:57.368899+0000","last_active":"2024-09-21T23:10:05.361810+0000","last_peered":"2024-09-21T23:10:05.361810+0000","last_clean":"2024-09-21T23:10:05.361810+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:10:05.361810+0000","last_undegraded":"2024-09-21T23:10:05.361810+0000","last_fullsized":"2024-09-21T23:10:05.361810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:47:00.401423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014237099999999999,"stat_sum":{"num_bytes":1277825,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":9,"num_read_kb":47,"num_write":6,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"29'5","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420570+0000","last_change":"2024-09-21T23:09:57.379066+0000","last_active":"2024-09-21T23:10:05.420570+0000","last_peered":"2024-09-21T23:10:05.420570+0000","last_clean":"2024-09-21T23:10:05.420570+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:10:05.420570+0000","last_undegraded":"2024-09-21T23:10:05.420570+0000","last_fullsized":"2024-09-21T23:10:05.420570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:26:40.678807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050204199999999996,"stat_sum":{"num_bytes":2293760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":4,"num_read_kb":0,"num_write":8,"num_write_kb":2057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"27'13","reported_seq":65,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.366844+0000","last_change":"2024-09-21T23:09:57.392519+0000","last_active":"2024-09-21T23:10:05.366844+0000","last_peered":"2024-09-21T23:10:05.366844+0000","last_clean":"2024-09-21T23:10:05.366844+0000","last_became_active":"2024-09-21T23:09:55.393358+0000","last_became_peered":"2024-09-21T23:09:55.393358+0000","last_unstale":"2024-09-21T23:10:05.366844+0000","last_undegraded":"2024-09-21T23:10:05.366844+0000","last_fullsized":"2024-09-21T23:10:05.366844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:12.034123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010715919999999999,"stat_sum":{"num_bytes":25165824,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":20,"num_read_kb":51,"num_write":11,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"29'23","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.366260+0000","last_change":"2024-09-21T23:09:57.392535+0000","last_active":"2024-09-21T23:10:05.366260+0000","last_peered":"2024-09-21T23:10:05.366260+0000","last_clean":"2024-09-21T23:10:05.366260+0000","last_became_active":"2024-09-21T23:09:55.391456+0000","last_became_peered":"2024-09-21T23:09:55.391456+0000","last_unstale":"2024-09-21T23:10:05.366260+0000","last_undegraded":"2024-09-21T23:10:05.366260+0000","last_fullsized":"2024-09-21T23:10:05.366260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:01:31.804323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010461890000000001,"stat_sum":{"num_bytes":24643759,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":336,"num_read_kb":424,"num_write":22,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"30'95","reported_seq":150,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.517065+0000","last_change":"2024-09-21T23:09:57.392849+0000","last_active":"2024-09-21T23:10:05.517065+0000","last_peered":"2024-09-21T23:10:05.517065+0000","last_clean":"2024-09-21T23:10:05.517065+0000","last_became_active":"2024-09-21T23:09:55.394946+0000","last_became_peered":"2024-09-21T23:09:55.394946+0000","last_unstale":"2024-09-21T23:10:05.517065+0000","last_undegraded":"2024-09-21T23:10:05.517065+0000","last_fullsized":"2024-09-21T23:10:05.517065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:09:01.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010405119999999999,"stat_sum":{"num_bytes":1048655,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":483,"num_read_kb":372,"num_write":162,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"30'28","reported_seq":411,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.521856+0000","last_change":"2024-09-21T23:09:57.494767+0000","last_active":"2024-09-21T23:10:05.521856+0000","last_peered":"2024-09-21T23:10:05.521856+0000","last_clean":"2024-09-21T23:10:05.521856+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:10:05.521856+0000","last_undegraded":"2024-09-21T23:10:05.521856+0000","last_fullsized":"2024-09-21T23:10:05.521856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:42:48.155765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071189699999999996,"stat_sum":{"num_bytes":32930100,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":509,"num_read_kb":716,"num_write":29,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"30'4","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.505039+0000","last_change":"2024-09-21T23:09:57.391982+0000","last_active":"2024-09-21T23:10:05.505039+0000","last_peered":"2024-09-21T23:10:05.505039+0000","last_clean":"2024-09-21T23:10:05.505039+0000","last_became_active":"2024-09-21T23:09:55.390030+0000","last_became_peered":"2024-09-21T23:09:55.390030+0000","last_unstale":"2024-09-21T23:10:05.505039+0000","last_undegraded":"2024-09-21T23:10:05.505039+0000","last_fullsized":"2024-09-21T23:10:05.505039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:09.972342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010743459999999999,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":45,"num_write":16,"num_write_kb":3406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"29'27","reported_seq":213,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381327+0000","last_change":"2024-09-21T23:09:57.385100+0000","last_active":"2024-09-21T23:10:05.381327+0000","last_peered":"2024-09-21T23:10:05.381327+0000","last_clean":"2024-09-21T23:10:05.381327+0000","last_became_active":"2024-09-21T23:09:55.382430+0000","last_became_peered":"2024-09-21T23:09:55.382430+0000","last_unstale":"2024-09-21T23:10:05.381327+0000","last_undegraded":"2024-09-21T23:10:05.381327+0000","last_fullsized":"2024-09-21T23:10:05.381327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:06:27.664101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021982479999999999,"stat_sum":{"num_bytes":32803,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":475,"num_read_kb":437,"num_write":23,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"29'8","reported_seq":118,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.508817+0000","last_change":"2024-09-21T23:09:57.392347+0000","last_active":"2024-09-21T23:10:05.508817+0000","last_peered":"2024-09-21T23:10:05.508817+0000","last_clean":"2024-09-21T23:10:05.508817+0000","last_became_active":"2024-09-21T23:09:55.390621+0000","last_became_peered":"2024-09-21T23:09:55.390621+0000","last_unstale":"2024-09-21T23:10:05.508817+0000","last_undegraded":"2024-09-21T23:10:05.508817+0000","last_fullsized":"2024-09-21T23:10:05.508817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:21.838918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":1024682,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":83,"num_read_kb":99,"num_write":7,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":282504314,"num_objects":86,"num_object_clones":16,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":21,"num_read":4367,"num_read_kb":5664,"num_write":714,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33378304,"data_stored":33026752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607,"ondisk_log_size":607,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1765376,"data_stored":1541394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":679936,"data_stored":450667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":4784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":4898816,"data_stored":4636854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1024000,"data_stored":796106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":643072,"data_stored":405980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":413696,"data_stored":212204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":247734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28584,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343256,"statfs":{"total":96636764160,"available":96607494144,"internally_reserved":0,"allocated":1589248,"data_stored":1300641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4091904,"data_stored":4009974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5758976,"data_stored":5725104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2170880,"data_stored":2160379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4399104,"data_stored":4351758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7651328,"data_stored":7610028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4579328,"data_stored":4512851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":356216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4329472,"data_stored":4300442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:10:07.793 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:10:07.839 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:07.838+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:07.862 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:07.862 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:07.883 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:07.888 INFO:teuthology.orchestra.run.smithi150.stdout:612 write 0xb3f4792 thru 0xb3fb060 (0x68cf bytes) 2024-09-21T23:10:07.894 INFO:teuthology.orchestra.run.smithi150.stdout:614 write 0x657e5f1 thru 0x6586f5e (0x896e bytes) 2024-09-21T23:10:07.901 INFO:teuthology.orchestra.run.smithi150.stdout:615 read 0x732298a thru 0x732b8f1 (0x8f68 bytes) 2024-09-21T23:10:07.901 INFO:teuthology.orchestra.run.smithi150.stdout:616 trunc from 0xb3fb061 to 0xab18fdb 2024-09-21T23:10:07.917 INFO:teuthology.orchestra.run.smithi150.stdout:617 clone 7 order 24 su 262144 sc 13 2024-09-21T23:10:07.938 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:07.934+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:08.006 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:08.007 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:08.020 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:08.039 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:08.038+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:10:08.132 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:10:08.139 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:08.138+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:08.148 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-21T23:10:07.441784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":283094701,"num_objects":90,"num_object_clones":16,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":21,"num_read":4483,"num_read_kb":5764,"num_write":875,"num_write_kb":16146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":727,"ondisk_log_size":727,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231652,"kb_used_data":11216,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754743068,"statfs":{"total":773094113280,"available":772856901632,"internally_reserved":0,"allocated":11485184,"data_stored":9591580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":280986635,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":21,"num_read":3981,"num_read_kb":5346,"num_write":667,"num_write_kb":13556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039586"},"pg_stats":[{"pgid":"3.1f","version":"24'1","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381528+0000","last_change":"2024-09-21T23:09:57.384622+0000","last_active":"2024-09-21T23:10:05.381528+0000","last_peered":"2024-09-21T23:10:05.381528+0000","last_clean":"2024-09-21T23:10:05.381528+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:10:05.381528+0000","last_undegraded":"2024-09-21T23:10:05.381528+0000","last_fullsized":"2024-09-21T23:10:05.381528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:12.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018862200000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":37,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362348+0000","last_change":"2024-09-21T23:09:57.368121+0000","last_active":"2024-09-21T23:10:05.362348+0000","last_peered":"2024-09-21T23:10:05.362348+0000","last_clean":"2024-09-21T23:10:05.362348+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:10:05.362348+0000","last_undegraded":"2024-09-21T23:10:05.362348+0000","last_fullsized":"2024-09-21T23:10:05.362348+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:20:19.079775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010293570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"30'24","reported_seq":110,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420340+0000","last_change":"2024-09-21T23:09:57.379419+0000","last_active":"2024-09-21T23:10:05.420340+0000","last_peered":"2024-09-21T23:10:05.420340+0000","last_clean":"2024-09-21T23:10:05.420340+0000","last_became_active":"2024-09-21T23:09:55.383016+0000","last_became_peered":"2024-09-21T23:09:55.383016+0000","last_unstale":"2024-09-21T23:10:05.420340+0000","last_undegraded":"2024-09-21T23:10:05.420340+0000","last_fullsized":"2024-09-21T23:10:05.420340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:31:19.672645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057415099999999996,"stat_sum":{"num_bytes":25690112,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":62,"num_read_kb":229,"num_write":22,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"29'16","reported_seq":69,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420235+0000","last_change":"2024-09-21T23:09:57.380029+0000","last_active":"2024-09-21T23:10:05.420235+0000","last_peered":"2024-09-21T23:10:05.420235+0000","last_clean":"2024-09-21T23:10:05.420235+0000","last_became_active":"2024-09-21T23:09:55.385836+0000","last_became_peered":"2024-09-21T23:09:55.385836+0000","last_unstale":"2024-09-21T23:10:05.420235+0000","last_undegraded":"2024-09-21T23:10:05.420235+0000","last_fullsized":"2024-09-21T23:10:05.420235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:03.351360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090318899999999999,"stat_sum":{"num_bytes":811057,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":126,"num_write":14,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"29'2","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381457+0000","last_change":"2024-09-21T23:09:57.383801+0000","last_active":"2024-09-21T23:10:05.381457+0000","last_peered":"2024-09-21T23:10:05.381457+0000","last_clean":"2024-09-21T23:10:05.381457+0000","last_became_active":"2024-09-21T23:09:55.377005+0000","last_became_peered":"2024-09-21T23:09:55.377005+0000","last_unstale":"2024-09-21T23:10:05.381457+0000","last_undegraded":"2024-09-21T23:10:05.381457+0000","last_fullsized":"2024-09-21T23:10:05.381457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:14:18.281291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007694,"stat_sum":{"num_bytes":61,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"30'85","reported_seq":233,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460701+0000","last_change":"2024-09-21T23:09:57.369613+0000","last_active":"2024-09-21T23:10:05.460701+0000","last_peered":"2024-09-21T23:10:05.460701+0000","last_clean":"2024-09-21T23:10:05.460701+0000","last_became_active":"2024-09-21T23:09:55.378756+0000","last_became_peered":"2024-09-21T23:09:55.378756+0000","last_unstale":"2024-09-21T23:10:05.460701+0000","last_undegraded":"2024-09-21T23:10:05.460701+0000","last_fullsized":"2024-09-21T23:10:05.460701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:59.643930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011201469999999999,"stat_sum":{"num_bytes":49,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":480,"num_read_kb":399,"num_write":129,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.459911+0000","last_change":"2024-09-21T23:09:57.374916+0000","last_active":"2024-09-21T23:10:05.459911+0000","last_peered":"2024-09-21T23:10:05.459911+0000","last_clean":"2024-09-21T23:10:05.459911+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:10:05.459911+0000","last_undegraded":"2024-09-21T23:10:05.459911+0000","last_fullsized":"2024-09-21T23:10:05.459911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:42.428614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073844600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"29'5","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361704+0000","last_change":"2024-09-21T23:09:57.367436+0000","last_active":"2024-09-21T23:10:05.361704+0000","last_peered":"2024-09-21T23:10:05.361704+0000","last_clean":"2024-09-21T23:10:05.361704+0000","last_became_active":"2024-09-21T23:09:55.366388+0000","last_became_peered":"2024-09-21T23:09:55.366388+0000","last_unstale":"2024-09-21T23:10:05.361704+0000","last_undegraded":"2024-09-21T23:10:05.361704+0000","last_fullsized":"2024-09-21T23:10:05.361704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:28.195405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054973999999999995,"stat_sum":{"num_bytes":2339902,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"30'48","reported_seq":119,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.506544+0000","last_change":"2024-09-21T23:09:57.384681+0000","last_active":"2024-09-21T23:10:05.506544+0000","last_peered":"2024-09-21T23:10:05.506544+0000","last_clean":"2024-09-21T23:10:05.506544+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:10:05.506544+0000","last_undegraded":"2024-09-21T23:10:05.506544+0000","last_fullsized":"2024-09-21T23:10:05.506544+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:46:55.092696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001819836,"stat_sum":{"num_bytes":27011594,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":147,"num_read_kb":328,"num_write":53,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"29'3","reported_seq":281,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.519118+0000","last_change":"2024-09-21T23:09:57.379117+0000","last_active":"2024-09-21T23:10:05.519118+0000","last_peered":"2024-09-21T23:10:05.519118+0000","last_clean":"2024-09-21T23:10:05.519118+0000","last_became_active":"2024-09-21T23:09:55.388499+0000","last_became_peered":"2024-09-21T23:09:55.388499+0000","last_unstale":"2024-09-21T23:10:05.519118+0000","last_undegraded":"2024-09-21T23:10:05.519118+0000","last_fullsized":"2024-09-21T23:10:05.519118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:11:30.098231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049884500000000004,"stat_sum":{"num_bytes":26451551,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":246,"num_read_kb":416,"num_write":5,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'82","reported_seq":529,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.437082+0000","last_change":"2024-09-21T23:09:57.392268+0000","last_active":"2024-09-21T23:10:05.437082+0000","last_peered":"2024-09-21T23:10:05.437082+0000","last_clean":"2024-09-21T23:10:05.437082+0000","last_became_active":"2024-09-21T23:09:55.384020+0000","last_became_peered":"2024-09-21T23:09:55.384020+0000","last_unstale":"2024-09-21T23:10:05.437082+0000","last_undegraded":"2024-09-21T23:10:05.437082+0000","last_fullsized":"2024-09-21T23:10:05.437082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:56.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313919,"stat_sum":{"num_bytes":25024047,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":1181,"num_read_kb":1030,"num_write":73,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"30'20","reported_seq":69,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.422446+0000","last_change":"2024-09-21T23:09:57.369003+0000","last_active":"2024-09-21T23:10:05.422446+0000","last_peered":"2024-09-21T23:10:05.422446+0000","last_clean":"2024-09-21T23:10:05.422446+0000","last_became_active":"2024-09-21T23:09:55.389660+0000","last_became_peered":"2024-09-21T23:09:55.389660+0000","last_unstale":"2024-09-21T23:10:05.422446+0000","last_undegraded":"2024-09-21T23:10:05.422446+0000","last_fullsized":"2024-09-21T23:10:05.422446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:52.209890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110800000000001,"stat_sum":{"num_bytes":3384830,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":25,"num_read_kb":50,"num_write":16,"num_write_kb":359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.374419+0000","last_change":"2024-09-21T23:09:48.068366+0000","last_active":"2024-09-21T23:10:05.374419+0000","last_peered":"2024-09-21T23:10:05.374419+0000","last_clean":"2024-09-21T23:10:05.374419+0000","last_became_active":"2024-09-21T23:09:45.718621+0000","last_became_peered":"2024-09-21T23:09:45.718621+0000","last_unstale":"2024-09-21T23:10:05.374419+0000","last_undegraded":"2024-09-21T23:10:05.374419+0000","last_fullsized":"2024-09-21T23:10:05.374419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:44:48.782244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'6","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381715+0000","last_change":"2024-09-21T23:09:57.383456+0000","last_active":"2024-09-21T23:10:05.381715+0000","last_peered":"2024-09-21T23:10:05.381715+0000","last_clean":"2024-09-21T23:10:05.381715+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:10:05.381715+0000","last_undegraded":"2024-09-21T23:10:05.381715+0000","last_fullsized":"2024-09-21T23:10:05.381715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:19:39.549662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069300299999999998,"stat_sum":{"num_bytes":32635978,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":83,"num_write":5,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.459944+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:10:05.459944+0000","last_peered":"2024-09-21T23:10:05.459944+0000","last_clean":"2024-09-21T23:10:05.459944+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:10:05.459944+0000","last_undegraded":"2024-09-21T23:10:05.459944+0000","last_fullsized":"2024-09-21T23:10:05.459944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"29'18","reported_seq":82,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362657+0000","last_change":"2024-09-21T23:09:57.368949+0000","last_active":"2024-09-21T23:10:05.362657+0000","last_peered":"2024-09-21T23:10:05.362657+0000","last_clean":"2024-09-21T23:10:05.362657+0000","last_became_active":"2024-09-21T23:09:55.382399+0000","last_became_peered":"2024-09-21T23:09:55.382399+0000","last_unstale":"2024-09-21T23:10:05.362657+0000","last_undegraded":"2024-09-21T23:10:05.362657+0000","last_fullsized":"2024-09-21T23:10:05.362657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:10:33.832274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333359999999999,"stat_sum":{"num_bytes":25482042,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":45,"num_read_kb":460,"num_write":15,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.367194+0000","last_change":"2024-09-21T23:09:48.068555+0000","last_active":"2024-09-21T23:10:05.367194+0000","last_peered":"2024-09-21T23:10:05.367194+0000","last_clean":"2024-09-21T23:10:05.367194+0000","last_became_active":"2024-09-21T23:09:45.869065+0000","last_became_peered":"2024-09-21T23:09:45.869065+0000","last_unstale":"2024-09-21T23:10:05.367194+0000","last_undegraded":"2024-09-21T23:10:05.367194+0000","last_fullsized":"2024-09-21T23:10:05.367194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:48.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088130999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"30'18","reported_seq":59,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460071+0000","last_change":"2024-09-21T23:09:57.375144+0000","last_active":"2024-09-21T23:10:05.460071+0000","last_peered":"2024-09-21T23:10:05.460071+0000","last_clean":"2024-09-21T23:10:05.460071+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:10:05.460071+0000","last_undegraded":"2024-09-21T23:10:05.460071+0000","last_fullsized":"2024-09-21T23:10:05.460071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:41.785855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079926499999999996,"stat_sum":{"num_bytes":4521984,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":85,"num_write":18,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460213+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:10:05.460213+0000","last_peered":"2024-09-21T23:10:05.460213+0000","last_clean":"2024-09-21T23:10:05.460213+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:10:05.460213+0000","last_undegraded":"2024-09-21T23:10:05.460213+0000","last_fullsized":"2024-09-21T23:10:05.460213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"29'6","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420367+0000","last_change":"2024-09-21T23:09:57.378889+0000","last_active":"2024-09-21T23:10:05.420367+0000","last_peered":"2024-09-21T23:10:05.420367+0000","last_clean":"2024-09-21T23:10:05.420367+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:10:05.420367+0000","last_undegraded":"2024-09-21T23:10:05.420367+0000","last_fullsized":"2024-09-21T23:10:05.420367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:45:17.033071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000364898,"stat_sum":{"num_bytes":248866,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":9,"num_read_kb":2,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'12","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460183+0000","last_change":"2024-09-21T23:09:57.368994+0000","last_active":"2024-09-21T23:10:05.460183+0000","last_peered":"2024-09-21T23:10:05.460183+0000","last_clean":"2024-09-21T23:10:05.460183+0000","last_became_active":"2024-09-21T23:09:55.377435+0000","last_became_peered":"2024-09-21T23:09:55.377435+0000","last_unstale":"2024-09-21T23:10:05.460183+0000","last_undegraded":"2024-09-21T23:10:05.460183+0000","last_fullsized":"2024-09-21T23:10:05.460183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:18:14.697224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060347799999999996,"stat_sum":{"num_bytes":5570560,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":12,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420742+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:10:05.420742+0000","last_peered":"2024-09-21T23:10:05.420742+0000","last_clean":"2024-09-21T23:10:05.420742+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:10:05.420742+0000","last_undegraded":"2024-09-21T23:10:05.420742+0000","last_fullsized":"2024-09-21T23:10:05.420742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"29'4","reported_seq":47,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.460007+0000","last_change":"2024-09-21T23:09:57.374883+0000","last_active":"2024-09-21T23:10:05.460007+0000","last_peered":"2024-09-21T23:10:05.460007+0000","last_clean":"2024-09-21T23:10:05.460007+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:10:05.460007+0000","last_undegraded":"2024-09-21T23:10:05.460007+0000","last_fullsized":"2024-09-21T23:10:05.460007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:34.370102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060209499999999997,"stat_sum":{"num_bytes":4543637,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":109,"num_write":4,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361290+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:10:05.361290+0000","last_peered":"2024-09-21T23:10:05.361290+0000","last_clean":"2024-09-21T23:10:05.361290+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:10:05.361290+0000","last_undegraded":"2024-09-21T23:10:05.361290+0000","last_fullsized":"2024-09-21T23:10:05.361290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"29'3","reported_seq":43,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381224+0000","last_change":"2024-09-21T23:09:57.385279+0000","last_active":"2024-09-21T23:10:05.381224+0000","last_peered":"2024-09-21T23:10:05.381224+0000","last_clean":"2024-09-21T23:10:05.381224+0000","last_became_active":"2024-09-21T23:09:55.376738+0000","last_became_peered":"2024-09-21T23:09:55.376738+0000","last_unstale":"2024-09-21T23:10:05.381224+0000","last_undegraded":"2024-09-21T23:10:05.381224+0000","last_fullsized":"2024-09-21T23:10:05.381224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:10:35.904378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021610209999999999,"stat_sum":{"num_bytes":45,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":5,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.365860+0000","last_change":"2024-09-21T23:09:48.068542+0000","last_active":"2024-09-21T23:10:05.365860+0000","last_peered":"2024-09-21T23:10:05.365860+0000","last_clean":"2024-09-21T23:10:05.365860+0000","last_became_active":"2024-09-21T23:09:45.714516+0000","last_became_peered":"2024-09-21T23:09:45.714516+0000","last_unstale":"2024-09-21T23:10:05.365860+0000","last_undegraded":"2024-09-21T23:10:05.365860+0000","last_fullsized":"2024-09-21T23:10:05.365860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:51.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000999572,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"29'18","reported_seq":61,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362145+0000","last_change":"2024-09-21T23:09:57.368695+0000","last_active":"2024-09-21T23:10:05.362145+0000","last_peered":"2024-09-21T23:10:05.362145+0000","last_clean":"2024-09-21T23:10:05.362145+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:10:05.362145+0000","last_undegraded":"2024-09-21T23:10:05.362145+0000","last_fullsized":"2024-09-21T23:10:05.362145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:40.264877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013976220000000001,"stat_sum":{"num_bytes":2293820,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":45,"num_write":26,"num_write_kb":2035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.419992+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:10:05.419992+0000","last_peered":"2024-09-21T23:10:05.419992+0000","last_clean":"2024-09-21T23:10:05.419992+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:10:05.419992+0000","last_undegraded":"2024-09-21T23:10:05.419992+0000","last_fullsized":"2024-09-21T23:10:05.419992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.366044+0000","last_change":"2024-09-21T23:09:41.675036+0000","last_active":"2024-09-21T23:10:05.366044+0000","last_peered":"2024-09-21T23:10:05.366044+0000","last_clean":"2024-09-21T23:10:05.366044+0000","last_became_active":"2024-09-21T23:09:41.674018+0000","last_became_peered":"2024-09-21T23:09:41.674018+0000","last_unstale":"2024-09-21T23:10:05.366044+0000","last_undegraded":"2024-09-21T23:10:05.366044+0000","last_fullsized":"2024-09-21T23:10:05.366044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:10.788395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"29'1","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361619+0000","last_change":"2024-09-21T23:09:57.367811+0000","last_active":"2024-09-21T23:10:05.361619+0000","last_peered":"2024-09-21T23:10:05.361619+0000","last_clean":"2024-09-21T23:10:05.361619+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:10:05.361619+0000","last_undegraded":"2024-09-21T23:10:05.361619+0000","last_fullsized":"2024-09-21T23:10:05.361619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:49:38.489900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094978,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"29'10","reported_seq":61,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.362724+0000","last_change":"2024-09-21T23:09:57.369343+0000","last_active":"2024-09-21T23:10:05.362724+0000","last_peered":"2024-09-21T23:10:05.362724+0000","last_clean":"2024-09-21T23:10:05.362724+0000","last_became_active":"2024-09-21T23:09:55.767547+0000","last_became_peered":"2024-09-21T23:09:55.767547+0000","last_unstale":"2024-09-21T23:10:05.362724+0000","last_undegraded":"2024-09-21T23:10:05.362724+0000","last_fullsized":"2024-09-21T23:10:05.362724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:20.771000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084957200000000004,"stat_sum":{"num_bytes":2342126,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":19,"num_read_kb":34,"num_write":10,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"29'12","reported_seq":83,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.374433+0000","last_change":"2024-09-21T23:09:57.494957+0000","last_active":"2024-09-21T23:10:05.374433+0000","last_peered":"2024-09-21T23:10:05.374433+0000","last_clean":"2024-09-21T23:10:05.374433+0000","last_became_active":"2024-09-21T23:09:55.386143+0000","last_became_peered":"2024-09-21T23:09:55.386143+0000","last_unstale":"2024-09-21T23:10:05.374433+0000","last_undegraded":"2024-09-21T23:10:05.374433+0000","last_fullsized":"2024-09-21T23:10:05.374433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:53.517804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073170699999999995,"stat_sum":{"num_bytes":1540310,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":51,"num_read_kb":61,"num_write":10,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"29'8","reported_seq":46,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.361810+0000","last_change":"2024-09-21T23:09:57.368899+0000","last_active":"2024-09-21T23:10:05.361810+0000","last_peered":"2024-09-21T23:10:05.361810+0000","last_clean":"2024-09-21T23:10:05.361810+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:10:05.361810+0000","last_undegraded":"2024-09-21T23:10:05.361810+0000","last_fullsized":"2024-09-21T23:10:05.361810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:47:00.401423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014237099999999999,"stat_sum":{"num_bytes":1277825,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":9,"num_read_kb":47,"num_write":6,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"29'5","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.420570+0000","last_change":"2024-09-21T23:09:57.379066+0000","last_active":"2024-09-21T23:10:05.420570+0000","last_peered":"2024-09-21T23:10:05.420570+0000","last_clean":"2024-09-21T23:10:05.420570+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:10:05.420570+0000","last_undegraded":"2024-09-21T23:10:05.420570+0000","last_fullsized":"2024-09-21T23:10:05.420570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:26:40.678807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050204199999999996,"stat_sum":{"num_bytes":2293760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":4,"num_read_kb":0,"num_write":8,"num_write_kb":2057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"27'13","reported_seq":65,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.366844+0000","last_change":"2024-09-21T23:09:57.392519+0000","last_active":"2024-09-21T23:10:05.366844+0000","last_peered":"2024-09-21T23:10:05.366844+0000","last_clean":"2024-09-21T23:10:05.366844+0000","last_became_active":"2024-09-21T23:09:55.393358+0000","last_became_peered":"2024-09-21T23:09:55.393358+0000","last_unstale":"2024-09-21T23:10:05.366844+0000","last_undegraded":"2024-09-21T23:10:05.366844+0000","last_fullsized":"2024-09-21T23:10:05.366844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:12.034123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010715919999999999,"stat_sum":{"num_bytes":25165824,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":20,"num_read_kb":51,"num_write":11,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"29'23","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.366260+0000","last_change":"2024-09-21T23:09:57.392535+0000","last_active":"2024-09-21T23:10:05.366260+0000","last_peered":"2024-09-21T23:10:05.366260+0000","last_clean":"2024-09-21T23:10:05.366260+0000","last_became_active":"2024-09-21T23:09:55.391456+0000","last_became_peered":"2024-09-21T23:09:55.391456+0000","last_unstale":"2024-09-21T23:10:05.366260+0000","last_undegraded":"2024-09-21T23:10:05.366260+0000","last_fullsized":"2024-09-21T23:10:05.366260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:01:31.804323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010461890000000001,"stat_sum":{"num_bytes":24643759,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":336,"num_read_kb":424,"num_write":22,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"30'95","reported_seq":150,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.517065+0000","last_change":"2024-09-21T23:09:57.392849+0000","last_active":"2024-09-21T23:10:05.517065+0000","last_peered":"2024-09-21T23:10:05.517065+0000","last_clean":"2024-09-21T23:10:05.517065+0000","last_became_active":"2024-09-21T23:09:55.394946+0000","last_became_peered":"2024-09-21T23:09:55.394946+0000","last_unstale":"2024-09-21T23:10:05.517065+0000","last_undegraded":"2024-09-21T23:10:05.517065+0000","last_fullsized":"2024-09-21T23:10:05.517065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:09:01.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010405119999999999,"stat_sum":{"num_bytes":1048655,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":483,"num_read_kb":372,"num_write":162,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"30'28","reported_seq":411,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.521856+0000","last_change":"2024-09-21T23:09:57.494767+0000","last_active":"2024-09-21T23:10:05.521856+0000","last_peered":"2024-09-21T23:10:05.521856+0000","last_clean":"2024-09-21T23:10:05.521856+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:10:05.521856+0000","last_undegraded":"2024-09-21T23:10:05.521856+0000","last_fullsized":"2024-09-21T23:10:05.521856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:42:48.155765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071189699999999996,"stat_sum":{"num_bytes":32930100,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":509,"num_read_kb":716,"num_write":29,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"30'4","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.505039+0000","last_change":"2024-09-21T23:09:57.391982+0000","last_active":"2024-09-21T23:10:05.505039+0000","last_peered":"2024-09-21T23:10:05.505039+0000","last_clean":"2024-09-21T23:10:05.505039+0000","last_became_active":"2024-09-21T23:09:55.390030+0000","last_became_peered":"2024-09-21T23:09:55.390030+0000","last_unstale":"2024-09-21T23:10:05.505039+0000","last_undegraded":"2024-09-21T23:10:05.505039+0000","last_fullsized":"2024-09-21T23:10:05.505039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:53:09.972342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010743459999999999,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":45,"num_write":16,"num_write_kb":3406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"29'27","reported_seq":213,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.381327+0000","last_change":"2024-09-21T23:09:57.385100+0000","last_active":"2024-09-21T23:10:05.381327+0000","last_peered":"2024-09-21T23:10:05.381327+0000","last_clean":"2024-09-21T23:10:05.381327+0000","last_became_active":"2024-09-21T23:09:55.382430+0000","last_became_peered":"2024-09-21T23:09:55.382430+0000","last_unstale":"2024-09-21T23:10:05.381327+0000","last_undegraded":"2024-09-21T23:10:05.381327+0000","last_fullsized":"2024-09-21T23:10:05.381327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:06:27.664101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021982479999999999,"stat_sum":{"num_bytes":32803,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":475,"num_read_kb":437,"num_write":23,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"29'8","reported_seq":118,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T23:10:05.508817+0000","last_change":"2024-09-21T23:09:57.392347+0000","last_active":"2024-09-21T23:10:05.508817+0000","last_peered":"2024-09-21T23:10:05.508817+0000","last_clean":"2024-09-21T23:10:05.508817+0000","last_became_active":"2024-09-21T23:09:55.390621+0000","last_became_peered":"2024-09-21T23:09:55.390621+0000","last_unstale":"2024-09-21T23:10:05.508817+0000","last_undegraded":"2024-09-21T23:10:05.508817+0000","last_fullsized":"2024-09-21T23:10:05.508817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:21.838918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":1024682,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":83,"num_read_kb":99,"num_write":7,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":282504314,"num_objects":86,"num_object_clones":16,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":21,"num_read":4367,"num_read_kb":5664,"num_write":714,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33378304,"data_stored":33026752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":607,"ondisk_log_size":607,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1765376,"data_stored":1541394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":679936,"data_stored":450667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":4784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":4898816,"data_stored":4636854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1024000,"data_stored":796106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":643072,"data_stored":405980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":413696,"data_stored":212204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":247734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28584,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343256,"statfs":{"total":96636764160,"available":96607494144,"internally_reserved":0,"allocated":1589248,"data_stored":1300641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4091904,"data_stored":4009974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5758976,"data_stored":5725104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2170880,"data_stored":2160379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4399104,"data_stored":4351758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7651328,"data_stored":7610028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4579328,"data_stored":4512851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":356216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4329472,"data_stored":4300442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:10:08.150 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:10:08.151 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:10:08.151 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T23:10:08.151 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T23:10:08.151 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:10:08.178 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4902 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.99635922700000001, 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030793945, 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4902 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.37e-07 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179183+0000", 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9280000000000002e-06 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179192+0000", 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7020000000000007e-06 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179198+0000", 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6780000000000003e-06 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201407+0000", 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022208301999999999 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201500+0000", 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3231000000000002e-05 2024-09-21T23:10:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.209974+0000", 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084735669999999996 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4903 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2359296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.99633125199999995, 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031240032000000001, 2024-09-21T23:10:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4903 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179211+0000", 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6630000000000002e-06 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179220+0000", 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8480000000000007e-06 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179226+0000", 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2650000000000002e-06 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210016+0000", 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030790128 2024-09-21T23:10:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210066+0000", 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9713000000000003e-05 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210448+0000", 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038190499999999998 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4904 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.99630402500000004, 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031636297000000001, 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4904 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179238+0000", 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5349999999999999e-06 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179247+0000", 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6449999999999994e-06 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179260+0000", 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3431999999999999e-05 2024-09-21T23:10:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210480+0000", 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031220121 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210550+0000", 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0352000000000007e-05 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210871+0000", 2024-09-21T23:10:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000320701 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4905 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.99626886000000003, 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036964023999999998, 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4905 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179271+0000", 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179274+0000", 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1350000000000001e-06 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179283+0000", 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1779999999999992e-06 2024-09-21T23:10:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179290+0000", 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210903+0000", 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031613162 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210976+0000", 2024-09-21T23:10:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2591999999999999e-05 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216234+0000", 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052583309999999998 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4906 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.99623911700000001, 2024-09-21T23:10:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037394366999999998, 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4906 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179303+0000", 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8219999999999998e-06 2024-09-21T23:10:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179312+0000", 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6419999999999996e-06 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179319+0000", 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6449999999999999e-06 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216272+0000", 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036953142000000001 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216352+0000", 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0044999999999997e-05 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216694+0000", 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034238399999999999 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5224 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 131072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192626+0000", 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98291273199999996, 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029283586, 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5224 2024-09-21T23:10:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192626+0000", 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192626+0000", 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192627+0000", 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192630+0000", 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5499999999999999e-06 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192642+0000", 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1260000000000001e-05 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192650+0000", 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.220813+0000", 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.028162342 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.220896+0000", 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3475999999999998e-05 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.221910+0000", 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010135960000000001 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5225 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 589824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192662+0000", 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98287706100000005, 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029414375999999999, 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5225 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192662+0000", 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192662+0000", 2024-09-21T23:10:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192662+0000", 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192666+0000", 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.236e-06 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192677+0000", 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1172000000000001e-05 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192685+0000", 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.242e-06 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.221633+0000", 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.028948092000000002 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.221717+0000", 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4243e-05 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222076+0000", 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035877499999999998 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5226 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98284271899999998, 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029942795000000001, 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5226 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192697+0000", 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7199999999999998e-07 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192701+0000", 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8e-06 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192712+0000", 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1156e-05 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192724+0000", 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2551999999999999e-05 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222110+0000", 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029385620000000001 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222191+0000", 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1520000000000006e-05 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222639+0000", 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044747500000000002 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5227 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98280337900000003, 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030372671, 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5227 2024-09-21T23:10:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192736+0000", 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192742+0000", 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2260000000000002e-06 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192753+0000", 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0762e-05 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192761+0000", 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.14e-06 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222674+0000", 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029912979999999999 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222754+0000", 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9816999999999998e-05 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223108+0000", 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035405799999999998 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5228 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98276646499999998, 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0320895, 2024-09-21T23:10:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5228 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192773+0000", 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192776+0000", 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2380000000000002e-06 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192787+0000", 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0764999999999999e-05 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192795+0000", 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3000000000000002e-06 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223154+0000", 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030358672999999999 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223230+0000", 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6268999999999998e-05 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224862+0000", 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001631635 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5229 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98273228199999996, 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032496191000000001, 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5229 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5199999999999996e-07 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192810+0000", 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192820+0000", 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0132000000000001e-05 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192829+0000", 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5900000000000008e-06 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224912+0000", 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032083410999999999 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224985+0000", 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2455999999999999e-05 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225303+0000", 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031788999999999998 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5230 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98269760799999994, 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033015543000000001, 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5230 2024-09-21T23:10:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192842+0000", 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192845+0000", 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192856+0000", 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1008e-05 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192865+0000", 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225398+0000", 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032533617000000001 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225489+0000", 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1038999999999994e-05 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225857+0000", 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000367597 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5231 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98266368599999998, 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033349594000000003, 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5231 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192876+0000", 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192879+0000", 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2059999999999999e-06 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192890+0000", 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0702e-05 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192898+0000", 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3299999999999999e-06 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225903+0000", 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033004622999999997 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225981+0000", 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7990000000000004e-05 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226225+0000", 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024411499999999999 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5232 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3801088~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.98262971600000004, 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033795809000000003, 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5232 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192910+0000", 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1800000000000005e-07 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192913+0000", 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1669999999999999e-06 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192924+0000", 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0692e-05 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192936+0000", 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2414e-05 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226252+0000", 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03331626 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226309+0000", 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6733e-05 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226705+0000", 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00039582499999999999 2024-09-21T23:10:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5404 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.97795452299999996, 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029524524, 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5404 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197586+0000", 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197593+0000", 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1350000000000003e-06 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198348+0000", 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00075463500000000003 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198360+0000", 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800999999999999e-05 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226750+0000", 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.028390658999999999 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226800+0000", 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0052000000000001e-05 2024-09-21T23:10:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227109+0000", 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308382 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5406 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.97711646799999996, 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029727338999999998, 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5406 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198423+0000", 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7100000000000001e-07 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198427+0000", 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198436+0000", 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198446+0000", 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6360000000000006e-06 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227686+0000", 2024-09-21T23:10:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029240289999999999 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227764+0000", 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7676000000000006e-05 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228150+0000", 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038574300000000002 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5407 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.97708258999999997, 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030154378999999999, 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5407 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198457+0000", 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198460+0000", 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3409999999999999e-06 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198470+0000", 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198478+0000", 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7130000000000005e-06 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228216+0000", 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029738243000000001 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228290+0000", 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3461999999999996e-05 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228611+0000", 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032108100000000002 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5408 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.97705112400000005, 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030920828000000001, 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5408 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198491+0000", 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198501+0000", 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5719999999999992e-06 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198508+0000", 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2470000000000002e-06 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228720+0000", 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030211598999999999 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228851+0000", 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013098299999999999 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229409+0000", 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055781799999999996 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5411 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3440640~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.97499435000000001, 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029725732000000001, 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5411 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7100000000000002e-07 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200549+0000", 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200558+0000", 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8400000000000007e-06 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200573+0000", 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5006e-05 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229813+0000", 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029239580000000001 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229897+0000", 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3572999999999994e-05 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230270+0000", 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037366400000000001 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5412 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.97494611499999995, 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030113958, 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5412 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4099999999999998e-07 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200626+0000", 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2672999999999997e-05 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200637+0000", 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1248e-05 2024-09-21T23:10:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200646+0000", 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6559999999999998e-06 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230319+0000", 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029672765 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230378+0000", 2024-09-21T23:10:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8696999999999997e-05 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230707+0000", 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032917800000000001 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:08.267 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:08.268 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:08.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:08.270 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:08.270+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:08.370 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:08.366+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:08.401 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:08.401 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:08.401 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:08.401 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:08.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:08.469 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:08.469 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:08.471 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:08.470+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:08.472 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:10:08.486 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:08.486 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:08.502 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:08.520 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:08.520 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:08.520 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:08.540 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:08.540 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:08.541 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:08.541 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:08.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:08.558 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:08.558 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:08.576 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:08.576 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:08.579 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:08.580 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:08.601 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:08.602+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:08.673 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:08.673 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:08.673 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:08.670+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:08.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:08.775 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:08.770+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:08.853 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:08.853 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:08.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:08.913 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:08.914+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:08.977 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:08.974+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:08.984 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:08.984 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:08.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:09.078 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:09.074+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:09.161 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:09.161 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:09.175 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:09.221 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:09.222+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:09.280 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:09.278+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:09.297 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:09.298 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:09.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:09.380 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:09.378+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:09.456 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:10:09.478 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:09.478 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:09.480 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:09.478+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:09.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:09.581 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:09.578+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:09.608 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone6 from 0xab18fdb (overlap 0x23a980a) to 0x0 2024-09-21T23:10:09.619 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:09.619 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:09.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:09.682 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:09.682+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:09.783 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:09.782+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:09.826 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:09.826 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:09.851 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:09.895 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:09.894+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:09.984 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:09.982+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:09.989 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:09.990 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:10.006 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:10.091 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:10.090+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:10.098 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-21T23:10:10.123 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.123 INFO:teuthology.orchestra.run.smithi150.stdout:620 read 0x19f9ca9 thru 0x1a098f8 (0xfc50 bytes) 2024-09-21T23:10:10.126 INFO:teuthology.orchestra.run.smithi150.stdout:621 write 0x1cb5ce7 thru 0x1cc264c (0xc966 bytes) 2024-09-21T23:10:10.166 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.167 INFO:teuthology.orchestra.run.smithi150.stdout:623 read 0x41d72f6 thru 0x41de727 (0x7432 bytes) 2024-09-21T23:10:10.167 INFO:teuthology.orchestra.run.smithi150.stdout:625 punch from 0xa3f713 to 0xa3fb23, (0x410 bytes) 2024-09-21T23:10:10.167 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.179 INFO:teuthology.orchestra.run.smithi150.stdout:629 write 0xdcf8f44 thru 0xdd0020c (0x72c9 bytes) 2024-09-21T23:10:10.185 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:10.185 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:10.186 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:10.186+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:10.189 INFO:teuthology.orchestra.run.smithi150.stdout:630 read 0x99cf263 thru 0x99da49b (0xb239 bytes) 2024-09-21T23:10:10.189 INFO:teuthology.orchestra.run.smithi150.stdout:631 read 0xa1cc4ea thru 0xa1d0287 (0x3d9e bytes) 2024-09-21T23:10:10.189 INFO:teuthology.orchestra.run.smithi150.stdout:636 read 0xaf9f606 thru 0xafad040 (0xda3b bytes) 2024-09-21T23:10:10.194 INFO:teuthology.orchestra.run.smithi150.stdout:637 trunc from 0xdd0020d to 0xb233e99 2024-09-21T23:10:10.203 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:10.287 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:10.286+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:10.340 INFO:teuthology.orchestra.run.smithi150.stdout:638 trunc from 0xb233e99 to 0xaf4971e 2024-09-21T23:10:10.350 INFO:teuthology.orchestra.run.smithi150.stdout:639 flatten 2024-09-21T23:10:10.356 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.356 INFO:teuthology.orchestra.run.smithi150.stdout:641 read 0xadd2b17 thru 0xaddc835 (0x9d1f bytes) 2024-09-21T23:10:10.357 INFO:teuthology.orchestra.run.smithi150.stdout:643 writesame 0x34b46ae thru 0x34b9165 data_size 0x2(0x4ab8 bytes) 2024-09-21T23:10:10.362 INFO:teuthology.orchestra.run.smithi150.stdout:644 read 0x1e96bed thru 0x1e9b1d8 (0x45ec bytes) 2024-09-21T23:10:10.371 INFO:teuthology.orchestra.run.smithi150.stdout:645 trunc from 0xaf4971e to 0x5973287 2024-09-21T23:10:10.384 INFO:teuthology.orchestra.run.smithi150.stdout:646 writesame 0x30bc257 thru 0x30c8656 data_size 0x40(0xc400 bytes) 2024-09-21T23:10:10.387 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:10.386+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:10.405 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:10.405 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:10.406 INFO:teuthology.orchestra.run.smithi150.stdout:647 write 0xc3b7d49 thru 0xc3ba6b4 (0x296c bytes) 2024-09-21T23:10:10.408 INFO:teuthology.orchestra.run.smithi150.stdout:648 punch from 0x6d3466a to 0x6d3a0de, (0x5a74 bytes) 2024-09-21T23:10:10.408 INFO:teuthology.orchestra.run.smithi150.stdout:649 read 0x1c6a1e7 thru 0x1c73dd2 (0x9bec bytes) 2024-09-21T23:10:10.409 INFO:teuthology.orchestra.run.smithi150.stdout:651 writesame 0x2d0ce58 thru 0x2d1616f data_size 0x24(0x9318 bytes) 2024-09-21T23:10:10.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:10.425 INFO:teuthology.orchestra.run.smithi150.stdout:652 trunc from 0xc3ba6b5 to 0x3d59699 2024-09-21T23:10:10.441 INFO:teuthology.orchestra.run.smithi150.stdout:653 read 0x11529f2 thru 0x1153298 (0x8a7 bytes) 2024-09-21T23:10:10.441 INFO:teuthology.orchestra.run.smithi150.stdout:654 clone 8 order 21 su 262144 sc 4 2024-09-21T23:10:10.489 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:10.486+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:10.563 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:10.563 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:10.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:10.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:10.617 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-21T23:10:10.618 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:10.614+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:10.638 INFO:teuthology.orchestra.run.smithi150.stdout:655 write 0x8421f16 thru 0x8429cef (0x7dda bytes) 2024-09-21T23:10:10.643 INFO:teuthology.orchestra.run.smithi150.stdout:656 write 0x570f56c thru 0x5717892 (0x8327 bytes) 2024-09-21T23:10:10.662 INFO:teuthology.orchestra.run.smithi150.stdout:657 write 0xe5ecd03 thru 0xe5f7c67 (0xaf65 bytes) 2024-09-21T23:10:10.669 INFO:teuthology.orchestra.run.smithi150.stdout:659 read 0xa18a758 thru 0xa199b3b (0xf3e4 bytes) 2024-09-21T23:10:10.669 INFO:teuthology.orchestra.run.smithi150.stdout:660 writesame 0xb912553 thru 0xb91e114 data_size 0x3(0xbbc2 bytes) 2024-09-21T23:10:10.676 INFO:teuthology.orchestra.run.smithi150.stdout:662 read 0x7dd246e thru 0x7ddb35f (0x8ef2 bytes) 2024-09-21T23:10:10.676 INFO:teuthology.orchestra.run.smithi150.stdout:664 writesame 0x69a0312 thru 0x69a6bfe data_size 0x1(0x68ed bytes) 2024-09-21T23:10:10.690 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:10.686+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:10.706 INFO:teuthology.orchestra.run.smithi150.stdout:665 trunc from 0xe5f7c68 to 0x47e3212 2024-09-21T23:10:10.720 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.741 INFO:teuthology.orchestra.run.smithi150.stdout:667 write 0xb9dafb0 thru 0xb9e8be7 (0xdc38 bytes) 2024-09-21T23:10:10.747 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.747 INFO:teuthology.orchestra.run.smithi150.stdout:670 punch from 0x97a2416 to 0x97ac68b, (0xa275 bytes) 2024-09-21T23:10:10.747 INFO:teuthology.orchestra.run.smithi150.stdout:672 punch from 0x6fb4e8 to 0x702ec5, (0x79dd bytes) 2024-09-21T23:10:10.747 INFO:teuthology.orchestra.run.smithi150.stdout:675 read 0x2d4cba1 thru 0x2d540bc (0x751c bytes) 2024-09-21T23:10:10.747 INFO:teuthology.orchestra.run.smithi150.stdout:676 read 0x4e1c6e1 thru 0x4e26bde (0xa4fe bytes) 2024-09-21T23:10:10.753 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:10.753 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:10.764 INFO:teuthology.orchestra.run.smithi150.stdout:678 trunc from 0xb9e8be8 to 0x5b7efa 2024-09-21T23:10:10.783 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:10.788 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:10.788 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:10.790 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:10.790+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:10.801 INFO:teuthology.orchestra.run.smithi150.stdout:679 writesame 0xe31c9c thru 0xe41b42 data_size 0x43(0xfea7 bytes) 2024-09-21T23:10:10.801 INFO:teuthology.orchestra.run.smithi150.stdout:681 read 0xd85aba thru 0xd930c5 (0xd60c bytes) 2024-09-21T23:10:10.808 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:10.808 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:10.816 INFO:teuthology.orchestra.run.smithi150.stdout:682 write 0x75cae0b thru 0x75cec43 (0x3e39 bytes) 2024-09-21T23:10:10.820 INFO:teuthology.orchestra.run.smithi150.stdout:683 write 0x4b1d262 thru 0x4b2cbfb (0xf99a bytes) 2024-09-21T23:10:10.827 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:10.828 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:10.840 INFO:teuthology.orchestra.run.smithi150.stdout:684 writesame 0xcd6f3b2 thru 0xcd74a2e data_size 0x1(0x567d bytes) 2024-09-21T23:10:10.840 INFO:teuthology.orchestra.run.smithi150.stdout:686 write 0xca9d0b4 thru 0xca9f6b0 (0x25fd bytes) 2024-09-21T23:10:10.843 INFO:teuthology.orchestra.run.smithi150.stdout:687 writesame 0xc915f6a thru 0xc921c94 data_size 0x1(0xbd2b bytes) 2024-09-21T23:10:10.848 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:10.848 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:10.849 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:10.874 INFO:teuthology.orchestra.run.smithi150.stdout:688 read 0x1039439 thru 0x103b83b (0x2403 bytes) 2024-09-21T23:10:10.874 INFO:teuthology.orchestra.run.smithi150.stdout:689 write 0x8993d2e thru 0x89a0fc6 (0xd299 bytes) 2024-09-21T23:10:10.877 INFO:teuthology.orchestra.run.smithi150.stdout:691 read 0xf81139 thru 0xf8a8f8 (0x97c0 bytes) 2024-09-21T23:10:10.877 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.877 INFO:teuthology.orchestra.run.smithi150.stdout:693 punch from 0x5d97f4f to 0x5d99797, (0x1848 bytes) 2024-09-21T23:10:10.878 INFO:teuthology.orchestra.run.smithi150.stdout:695 read 0x59daea7 thru 0x59e472d (0x9887 bytes) 2024-09-21T23:10:10.878 INFO:teuthology.orchestra.run.smithi150.stdout:696 write 0xc7831fe thru 0xc78c0c9 (0x8ecc bytes) 2024-09-21T23:10:10.880 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:10.880 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:10.885 INFO:teuthology.orchestra.run.smithi150.stdout:697 punch from 0x42ea9ed to 0x42f50ab, (0xa6be bytes) 2024-09-21T23:10:10.885 INFO:teuthology.orchestra.run.smithi150.stdout:698 punch from 0xc1da22c to 0xc1e30ba, (0x8e8e bytes) 2024-09-21T23:10:10.885 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.885 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.891 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:10.890+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:10.892 INFO:teuthology.orchestra.run.smithi150.stdout:702 trunc from 0xcd74a2f to 0x7ced4e6 2024-09-21T23:10:10.903 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:10.920 INFO:teuthology.orchestra.run.smithi150.stdout:703 writesame 0xe88e07e thru 0xe892709 data_size 0x46(0x468c bytes) 2024-09-21T23:10:10.920 INFO:teuthology.orchestra.run.smithi150.stdout:704 read 0x9a0b23b thru 0x9a0e4f7 (0x32bd bytes) 2024-09-21T23:10:10.920 INFO:teuthology.orchestra.run.smithi150.stdout:705 read 0xe41d742 thru 0xe41e644 (0xf03 bytes) 2024-09-21T23:10:10.920 INFO:teuthology.orchestra.run.smithi150.stdout:707 write 0xb4075bb thru 0xb412696 (0xb0dc bytes) 2024-09-21T23:10:10.921 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:10.921 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:10.921 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:10.921 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:10.923 INFO:teuthology.orchestra.run.smithi150.stdout:708 writesame 0xdabc5f1 thru 0xdac6e87 data_size 0x1(0xa897 bytes) 2024-09-21T23:10:10.927 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:10.927 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:10.932 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:10.932 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:10.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:10.941 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:10.960 INFO:teuthology.orchestra.run.smithi150.stdout:711 trunc from 0xe89270a to 0xeccfa3 2024-09-21T23:10:10.972 INFO:teuthology.orchestra.run.smithi150.stdout:712 read 0x824d1f thru 0x82e462 (0x9744 bytes) 2024-09-21T23:10:10.990 INFO:teuthology.orchestra.run.smithi150.stdout:714 writesame 0x988a62a thru 0x9897df5 data_size 0x7(0xd7cc bytes) 2024-09-21T23:10:10.990 INFO:teuthology.orchestra.run.smithi150.stdout:715 punch from 0x5e700a3 to 0x5e77436, (0x7393 bytes) 2024-09-21T23:10:10.992 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:10.990+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:11.001 INFO:teuthology.orchestra.run.smithi150.stdout:716 writesame 0xd3c42ec thru 0xd3cd551 data_size 0x7(0x9266 bytes) 2024-09-21T23:10:11.001 INFO:teuthology.orchestra.run.smithi150.stdout:718 writesame 0xb671d13 thru 0xb673056 data_size 0x12(0x1344 bytes) 2024-09-21T23:10:11.005 INFO:teuthology.orchestra.run.smithi150.stdout:721 writesame 0x14f9480 thru 0x14fab7e data_size 0x7(0x16ff bytes) 2024-09-21T23:10:11.010 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:11.010 INFO:teuthology.orchestra.run.smithi150.stdout:724 read 0x1ed94 thru 0x21cfd (0x2f6a bytes) 2024-09-21T23:10:11.018 INFO:teuthology.orchestra.run.smithi150.stdout:725 write 0xdb6793a thru 0xdb6948a (0x1b51 bytes) 2024-09-21T23:10:11.021 INFO:teuthology.orchestra.run.smithi150.stdout:728 write 0x439e72a thru 0x43ad85b (0xf132 bytes) 2024-09-21T23:10:11.025 INFO:teuthology.orchestra.run.smithi150.stdout:731 writesame 0x832a306 thru 0x8330060 data_size 0x1(0x5d5b bytes) 2024-09-21T23:10:11.038 INFO:teuthology.orchestra.run.smithi150.stdout:732 punch from 0xaf3128b to 0xaf3db54, (0xc8c9 bytes) 2024-09-21T23:10:11.039 INFO:teuthology.orchestra.run.smithi150.stdout:733 read 0xc961fc7 thru 0xc965c7e (0x3cb8 bytes) 2024-09-21T23:10:11.039 INFO:teuthology.orchestra.run.smithi150.stdout:734 punch from 0x1455111 to 0x1456333, (0x1222 bytes) 2024-09-21T23:10:11.039 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:11.039 INFO:teuthology.orchestra.run.smithi150.stdout:736 writesame 0x741fc43 thru 0x7427b39 data_size 0x1(0x7ef7 bytes) 2024-09-21T23:10:11.056 INFO:teuthology.orchestra.run.smithi150.stdout:737 writesame 0xaab4045 thru 0xaab829d data_size 0x2b(0x4259 bytes) 2024-09-21T23:10:11.063 INFO:teuthology.orchestra.run.smithi150.stdout:738 trunc from 0xdb6948b to 0xe0253e8 2024-09-21T23:10:11.071 INFO:teuthology.orchestra.run.smithi150.stdout:739 writesame 0xa0acf4e thru 0xa0b9483 data_size 0x1(0xc536 bytes) 2024-09-21T23:10:11.072 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:11.072 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:11.072 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:11.073 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:11.073 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:11.088 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:11.131 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:11.130+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:11.144 INFO:teuthology.orchestra.run.smithi150.stdout:740 trunc from 0xe0253e8 to 0x90c197f 2024-09-21T23:10:11.157 INFO:teuthology.orchestra.run.smithi150.stdout:741 write 0x39c482f thru 0x39cc6ae (0x7e80 bytes) 2024-09-21T23:10:11.177 INFO:teuthology.orchestra.run.smithi150.stdout:742 write 0xe2836e7 thru 0xe293439 (0xfd53 bytes) 2024-09-21T23:10:11.182 INFO:teuthology.orchestra.run.smithi150.stdout:743 writesame 0x1dcbf52 thru 0x1dd47db data_size 0x1(0x888a bytes) 2024-09-21T23:10:11.193 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:11.194+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:11.206 INFO:teuthology.orchestra.run.smithi150.stdout:744 punch from 0x233ae12 to 0x2342b95, (0x7d83 bytes) 2024-09-21T23:10:11.206 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:11.206 INFO:teuthology.orchestra.run.smithi150.stdout:746 write 0xc6b5c54 thru 0xc6b94b6 (0x3863 bytes) 2024-09-21T23:10:11.211 INFO:teuthology.orchestra.run.smithi150.stdout:747 write 0x856c75e thru 0x8574153 (0x79f6 bytes) 2024-09-21T23:10:11.215 INFO:teuthology.orchestra.run.smithi150.stdout:750 write 0xb8916c thru 0xb8b1a9 (0x203e bytes) 2024-09-21T23:10:11.220 INFO:teuthology.orchestra.run.smithi150.stdout:751 punch from 0x2fe2347 to 0x2fe8c97, (0x6950 bytes) 2024-09-21T23:10:11.220 INFO:teuthology.orchestra.run.smithi150.stdout:752 write 0x56c72af thru 0x56d12bb (0xa00d bytes) 2024-09-21T23:10:11.224 INFO:teuthology.orchestra.run.smithi150.stdout:756 punch from 0x9008cd7 to 0x90151ca, (0xc4f3 bytes) 2024-09-21T23:10:11.224 INFO:teuthology.orchestra.run.smithi150.stdout:757 punch from 0xd0aceff to 0xd0ada9a, (0xb9b bytes) 2024-09-21T23:10:11.224 INFO:teuthology.orchestra.run.smithi150.stdout:760 read 0xbb63c thru 0xc1a30 (0x63f5 bytes) 2024-09-21T23:10:11.225 INFO:teuthology.orchestra.run.smithi150.stdout:761 writesame 0x740b22a thru 0x74144fc data_size 0x11(0x92d3 bytes) 2024-09-21T23:10:11.229 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:11.229 INFO:teuthology.orchestra.run.smithi150.stdout:763 write 0x3834e58 thru 0x3840e61 (0xc00a bytes) 2024-09-21T23:10:11.236 INFO:teuthology.orchestra.run.smithi150.stdout:764 read 0xf57f4f thru 0xf606d2 (0x8784 bytes) 2024-09-21T23:10:11.236 INFO:teuthology.orchestra.run.smithi150.stdout:765 writesame 0x1cb1b95 thru 0x1cbd957 data_size 0x1(0xbdc3 bytes) 2024-09-21T23:10:11.278 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:11.278 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:11.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.561011162, 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039594890000000001, 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716126+0000", 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716131+0000", 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716144+0000", 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2267e-05 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716158+0000", 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4731e-05 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716239+0000", 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1013999999999994e-05 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716271+0000", 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1535999999999998e-05 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720016+0000", 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037447399999999999 2024-09-21T23:10:11.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720084+0000", 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8813999999999995e-05 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:933 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'1, mlcod=0'0)", 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.3250209220000002, 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001140825, 2024-09-21T23:10:11.283 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952115+0000", 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952116+0000", 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952303+0000", 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187238 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952317+0000", 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4348000000000001e-05 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952326+0000", 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5439999999999999e-06 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952386+0000", 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0309000000000001e-05 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.952405+0000", 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9208999999999999e-05 2024-09-21T23:10:11.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.953214+0000", 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00080896899999999999 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.953256+0000", 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1714000000000001e-05 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:945 3.9 e27/21 3:96c84604:::rbd_data.1149b00fdffd.0000000000000006:head v 27'2, mlcod=0'0)", 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.3034750559999999, 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001850607, 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973661+0000", 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973663+0000", 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.728e-06 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973731+0000", 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7798000000000004e-05 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973749+0000", 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8703e-05 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973759+0000", 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7890000000000006e-06 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973897+0000", 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138422 2024-09-21T23:10:11.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.973941+0000", 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3192999999999998e-05 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.975430+0000", 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001489499 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.975512+0000", 2024-09-21T23:10:11.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1475000000000002e-05 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:947 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'3, mlcod=27'1)", 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2992211000000005, 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014565820000000001, 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977915+0000", 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977916+0000", 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:10:11.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977965+0000", 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8903000000000003e-05 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977979+0000", 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4579999999999999e-05 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.977987+0000", 2024-09-21T23:10:11.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3579999999999997e-06 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.978125+0000", 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013831600000000001 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.978170+0000", 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5346000000000003e-05 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.979284+0000", 2024-09-21T23:10:11.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011134579999999999 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.979372+0000", 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7773000000000001e-05 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:949 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'4, mlcod=27'2)", 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2801423350000007, 2024-09-21T23:10:11.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012844040000000001, 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996994+0000", 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.996995+0000", 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3480000000000001e-06 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997005+0000", 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4879999999999996e-06 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997036+0000", 2024-09-21T23:10:11.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1080000000000001e-05 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997046+0000", 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0338e-05 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997191+0000", 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000145069 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.997231+0000", 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0309000000000003e-05 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998169+0000", 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00093785299999999999 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998278+0000", 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010891900000000001 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:951 3.9 e27/21 3:96c84604:::rbd_data.1149b00fdffd.0000000000000006:head v 27'5, mlcod=27'2)", 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.2790643189999997, 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011579310000000001, 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998072+0000", 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998073+0000", 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6929999999999999e-06 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998107+0000", 2024-09-21T23:10:11.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.307e-05 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998132+0000", 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5420000000000001e-05 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998146+0000", 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4463e-05 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998324+0000", 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017724200000000001 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.998346+0000", 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2334e-05 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.999157+0000", 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081133300000000002 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:02.999230+0000", 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2376e-05 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:1073 3.9 e27/21 3:907ac6c4:::rbd_data.1149b00fdffd.0000000000000004:head v 27'7, mlcod=27'5)", 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0950737559999997, 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001363263, 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.305 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182062+0000", 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182065+0000", 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4569999999999999e-06 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182082+0000", 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6736e-05 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182113+0000", 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1412999999999999e-05 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182129+0000", 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5996999999999999e-05 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182268+0000", 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013877900000000001 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.182311+0000", 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2942999999999999e-05 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.183338+0000", 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010276619999999999 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:03.183426+0000", 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7275999999999996e-05 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2850 3.4 e30/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 30'30, mlcod=30'28)", 2024-09-21T23:10:11.309 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.251119739, 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033350860000000001, 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026017+0000", 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.68e-07 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026023+0000", 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1959999999999996e-06 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026033+0000", 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8120000000000002e-06 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026042+0000", 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7059999999999993e-06 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026127+0000", 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4973000000000001e-05 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026145+0000", 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8422000000000001e-05 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029303+0000", 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031578980000000001 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029351+0000", 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8411000000000001e-05 2024-09-21T23:10:11.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2900 3.4 e30/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 30'31, mlcod=30'29)", 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.136827+0000", 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1403092529999999, 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001321544, 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136827+0000", 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136827+0000", 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136828+0000", 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136847+0000", 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9488e-05 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136863+0000", 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5658000000000002e-05 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136872+0000", 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7269999999999996e-06 2024-09-21T23:10:11.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136999+0000", 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012732299999999999 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.137023+0000", 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3574000000000002e-05 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138097+0000", 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001074158 2024-09-21T23:10:11.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138148+0000", 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1666000000000003e-05 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2902 3.0 e30/21 3:02fe727d:::rbd_data.114912f5d0f.000000000000000d:head v 30'11, mlcod=30'9)", 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1402413559999998, 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012365950000000001, 2024-09-21T23:10:11.318 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136895+0000", 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136900+0000", 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7509999999999999e-06 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136909+0000", 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9129999999999992e-06 2024-09-21T23:10:11.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.136914+0000", 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5820000000000001e-06 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.137182+0000", 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026748899999999999 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.137197+0000", 2024-09-21T23:10:11.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5447999999999998e-05 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138071+0000", 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087370000000000004 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.138131+0000", 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0297000000000002e-05 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:3152 3.4 e31/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 31'33, mlcod=30'30)", 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6715401820000002, 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018085410000000001, 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605596+0000", 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3700000000000001e-07 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605604+0000", 2024-09-21T23:10:11.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6669999999999996e-06 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605627+0000", 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3167e-05 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605638+0000", 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0959e-05 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605704+0000", 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5925000000000003e-05 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.605728+0000", 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3765e-05 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.607361+0000", 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016327340000000001 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.607404+0000", 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3887000000000002e-05 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:3154 3.0 e31/21 3:02fe727d:::rbd_data.114912f5d0f.000000000000000d:head v 31'14, mlcod=30'11)", 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.608529+0000", 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6686074580000003, 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013675969999999999, 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608529+0000", 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608529+0000", 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608530+0000", 2024-09-21T23:10:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608537+0000", 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4959999999999999e-06 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608552+0000", 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5304000000000001e-05 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608560+0000", 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0620000000000001e-06 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608624+0000", 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3732000000000006e-05 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.608650+0000", 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5842000000000001e-05 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.609847+0000", 2024-09-21T23:10:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001196768 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.609896+0000", 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9496000000000002e-05 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:4287 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'16, mlcod=31'14)", 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.2477677270000003, 2024-09-21T23:10:11.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015395872, 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029371+0000", 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2759999999999999e-06 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031303+0000", 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019322359999999999 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031331+0000", 2024-09-21T23:10:11.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8408999999999999e-05 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031342+0000", 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1086e-05 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031428+0000", 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5990999999999997e-05 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031454+0000", 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044695+0000", 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013240784 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044764+0000", 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9623999999999996e-05 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.335 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:4299 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'17, mlcod=31'15)", 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.070721+0000", 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.2064145909999997, 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014312190000000001, 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070721+0000", 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070721+0000", 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070724+0000", 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.83e-06 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070742+0000", 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070782+0000", 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9752999999999997e-05 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070798+0000", 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6816999999999998e-05 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070878+0000", 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9777000000000005e-05 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.070901+0000", 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3006999999999998e-05 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.072107+0000", 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012055939999999999 2024-09-21T23:10:11.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.072153+0000", 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5998000000000002e-05 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:5384 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'20, mlcod=31'18)", 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.201914+0000", 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.0752220560000003, 2024-09-21T23:10:11.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021100590000000001, 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201914+0000", 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201914+0000", 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201915+0000", 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3999999999999999e-06 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201925+0000", 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0002e-05 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201948+0000", 2024-09-21T23:10:11.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2586e-05 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201958+0000", 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0076e-05 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.202048+0000", 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0012999999999994e-05 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.202093+0000", 2024-09-21T23:10:11.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4799000000000003e-05 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.203959+0000", 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001865853 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.204024+0000", 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.533e-05 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6012 3.4 e31/21 3:21794b57:::rbd_data.1149d3a069d9.0000000000000003:head v 31'34, mlcod=30'31)", 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.0087444909999999, 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094147550000000003, 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268393+0000", 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.72e-06 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269210+0000", 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081656700000000003 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269235+0000", 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5000000000000001e-05 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269248+0000", 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3393e-05 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269314+0000", 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6180000000000007e-05 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269341+0000", 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6554999999999999e-05 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277723+0000", 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083823960000000003 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277806+0000", 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2943999999999997e-05 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6167 3.4 e32/21 3:21794b57:::rbd_data.1149d3a069d9.0000000000000003:head v 32'36, mlcod=31'33)", 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.591699+0000", 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.6854366729999999, 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019956790000000002, 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591699+0000", 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591699+0000", 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591700+0000", 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1799999999999995e-07 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591707+0000", 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9469999999999997e-06 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591732+0000", 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4624999999999999e-05 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591740+0000", 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9379999999999998e-06 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591792+0000", 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1289000000000003e-05 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.591805+0000", 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3743e-05 2024-09-21T23:10:11.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.593652+0000", 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001846241 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.593695+0000", 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3377999999999997e-05 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6173 3.0 e32/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 32'22, mlcod=31'19)", 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.6787026049999998, 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002753003, 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598435+0000", 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8870000000000001e-06 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598445+0000", 2024-09-21T23:10:11.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4949999999999997e-06 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598464+0000", 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8720999999999999e-05 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598473+0000", 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5980000000000003e-06 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598540+0000", 2024-09-21T23:10:11.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7096999999999998e-05 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598564+0000", 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3288000000000001e-05 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601136+0000", 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025726099999999999 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601186+0000", 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0306999999999998e-05 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11716 3.9 3:94516a46:::rbd_data.1149e13ad964.000000000000006f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.34730326500000003, 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097894799999999997, 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11716 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929835+0000", 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2529999999999999e-06 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929846+0000", 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0859000000000001e-05 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929869+0000", 2024-09-21T23:10:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3450000000000001e-05 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929879+0000", 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7270000000000002e-06 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929973+0000", 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3917000000000002e-05 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.930106+0000", 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013346199999999999 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939479+0000", 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.009372755 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939522+0000", 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3025999999999998e-05 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939622+0000", 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010003100000000001 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11722 3.9 3:94516a46:::rbd_data.1149e13ad964.000000000000006f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:10.965367+0000", 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.311768977, 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001124353, 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11722 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965367+0000", 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965367+0000", 2024-09-21T23:10:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965369+0000", 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.778e-06 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965378+0000", 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5419999999999994e-06 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965400+0000", 2024-09-21T23:10:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1407e-05 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965409+0000", 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965489+0000", 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9698999999999997e-05 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.965586+0000", 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7220999999999996e-05 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.966353+0000", 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00076690700000000005 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.966406+0000", 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2651000000000001e-05 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.966491+0000", 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5514999999999997e-05 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.368 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:11.369 INFO:teuthology.orchestra.run.smithi150.stdout:766 trunc from 0xe29343a to 0x47d6ea3 2024-09-21T23:10:11.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:11.371 INFO:teuthology.orchestra.run.smithi150.stdout:767 writesame 0x833d90f thru 0x834328a data_size 0x5c(0x597c bytes) 2024-09-21T23:10:11.371 INFO:teuthology.orchestra.run.smithi150.stdout:768 write 0x380a4e1 thru 0x3819a19 (0xf539 bytes) 2024-09-21T23:10:11.371 INFO:teuthology.orchestra.run.smithi150.stdout:769 trunc from 0x834328b to 0x29408ae 2024-09-21T23:10:11.371 INFO:teuthology.orchestra.run.smithi150.stdout:770 punch from 0x1205423 to 0x12144bc, (0xf099 bytes) 2024-09-21T23:10:11.371 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:11.371 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:11.366+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:11.371 INFO:teuthology.orchestra.run.smithi150.stdout:774 trunc from 0x29408ae to 0xcf35fba 2024-09-21T23:10:11.379 INFO:teuthology.orchestra.run.smithi150.stdout:777 read 0x3651d7c thru 0x365d023 (0xb2a8 bytes) 2024-09-21T23:10:11.379 INFO:teuthology.orchestra.run.smithi150.stdout:778 write 0xb6d381f thru 0xb6d4158 (0x93a bytes) 2024-09-21T23:10:11.400 INFO:teuthology.orchestra.run.smithi150.stdout:779 trunc from 0xcf35fba to 0x455a592 2024-09-21T23:10:11.420 INFO:teuthology.orchestra.run.smithi150.stdout:780 trunc from 0x455a592 to 0x8892422 2024-09-21T23:10:11.427 INFO:teuthology.orchestra.run.smithi150.stdout:781 read 0x465d033 thru 0x4665d8b (0x8d59 bytes) 2024-09-21T23:10:11.427 INFO:teuthology.orchestra.run.smithi150.stdout:782 read 0x7f4d6ee thru 0x7f5bcd4 (0xe5e7 bytes) 2024-09-21T23:10:11.427 INFO:teuthology.orchestra.run.smithi150.stdout:783 read 0x4154de5 thru 0x4156590 (0x17ac bytes) 2024-09-21T23:10:11.427 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:11.435 INFO:teuthology.orchestra.run.smithi150.stdout:785 writesame 0x8f2ea74 thru 0x8f34305 data_size 0x3(0x5892 bytes) 2024-09-21T23:10:11.449 INFO:teuthology.orchestra.run.smithi150.stdout:786 writesame 0xc9c7b3c thru 0xc9cee05 data_size 0x7(0x72ca bytes) 2024-09-21T23:10:11.467 INFO:teuthology.orchestra.run.smithi150.stdout:787 trunc from 0xc9cee06 to 0x21f68c4 2024-09-21T23:10:11.469 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:11.466+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:11.485 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:10:11.493 INFO:teuthology.orchestra.run.smithi150.stdout:790 write 0x7d09f1b thru 0x7d1863f (0xe725 bytes) 2024-09-21T23:10:11.507 INFO:teuthology.orchestra.run.smithi150.stdout:791 write 0xa0244a8 thru 0xa0249a1 (0x4fa bytes) 2024-09-21T23:10:11.522 INFO:teuthology.orchestra.run.smithi150.stdout:792 write 0xccbad55 thru 0xccc13c1 (0x666d bytes) 2024-09-21T23:10:11.529 INFO:teuthology.orchestra.run.smithi150.stdout:793 write 0x135bdbd thru 0x135e891 (0x2ad5 bytes) 2024-09-21T23:10:11.533 INFO:teuthology.orchestra.run.smithi150.stdout:795 read 0x1435d03 thru 0x143d064 (0x7362 bytes) 2024-09-21T23:10:11.533 INFO:teuthology.orchestra.run.smithi150.stdout:796 write 0xb44f057 thru 0xb458cc1 (0x9c6b bytes) 2024-09-21T23:10:11.534 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:11971 3.19 e35/21 3:9f8fb82f:::rbd_object_map.1149e13ad964:head v 35'93, mlcod=35'91)", 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.536145+0000", 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00027657000000000002, 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030472700000000003, 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536145+0000", 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536145+0000", 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536146+0000", 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2500000000000004e-07 2024-09-21T23:10:11.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536152+0000", 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8340000000000004e-06 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536164+0000", 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1708999999999999e-05 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536170+0000", 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:10:11.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536234+0000", 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4647999999999995e-05 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.536265+0000", 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0488999999999999e-05 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.538 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:11.539 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 1 2024-09-21T23:10:11.539 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:11.539 INFO:teuthology.orchestra.run.smithi150.stdout:797 punch from 0x726b43c to 0x7270677, (0x523b bytes) 2024-09-21T23:10:11.539 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:11.539 INFO:teuthology.orchestra.run.smithi150.stdout:799 write 0x78c7eec thru 0x78d05f3 (0x8708 bytes) 2024-09-21T23:10:11.540 INFO:teuthology.orchestra.run.smithi150.stdout:800 punch from 0x79fd8c6 to 0x7a07454, (0x9b8e bytes) 2024-09-21T23:10:11.540 INFO:teuthology.orchestra.run.smithi150.stdout:801 read 0x2e35e09 thru 0x2e428a6 (0xca9e bytes) 2024-09-21T23:10:11.540 INFO:teuthology.orchestra.run.smithi150.stdout:803 writesame 0x15028a5 thru 0x150f47c data_size 0x2c(0xcbd8 bytes) 2024-09-21T23:10:11.545 INFO:teuthology.orchestra.run.smithi150.stdout:804 trunc from 0xccc13c2 to 0xd98aad6 2024-09-21T23:10:11.551 INFO:teuthology.orchestra.run.smithi150.stdout:805 clone 9 order 18 su 1024 sc 14 2024-09-21T23:10:11.560 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:11.603 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:11.602+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:11.671 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:11.670+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:11.724 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:11.724 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:11.724 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:11.724 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:11.724 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:11.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:11.783 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:11.782+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4902 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6808016229999998, 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030793945, 2024-09-21T23:10:11.860 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4902 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.37e-07 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179183+0000", 2024-09-21T23:10:11.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9280000000000002e-06 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179192+0000", 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7020000000000007e-06 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179198+0000", 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6780000000000003e-06 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201407+0000", 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022208301999999999 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201500+0000", 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3231000000000002e-05 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.209974+0000", 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084735669999999996 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4903 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2359296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6807736479999997, 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031240032000000001, 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4903 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179211+0000", 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6630000000000002e-06 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179220+0000", 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8480000000000007e-06 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179226+0000", 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2650000000000002e-06 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210016+0000", 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030790128 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210066+0000", 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9713000000000003e-05 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210448+0000", 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038190499999999998 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4904 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:11.869 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6807464210000003, 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031636297000000001, 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4904 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:11.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179238+0000", 2024-09-21T23:10:11.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5349999999999999e-06 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179247+0000", 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6449999999999994e-06 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179260+0000", 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3431999999999999e-05 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210480+0000", 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031220121 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210550+0000", 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0352000000000007e-05 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210871+0000", 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000320701 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4905 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6807112560000004, 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036964023999999998, 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4905 2024-09-21T23:10:11.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179271+0000", 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179274+0000", 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1350000000000001e-06 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179283+0000", 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1779999999999992e-06 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179290+0000", 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210903+0000", 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031613162 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210976+0000", 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2591999999999999e-05 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216234+0000", 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052583309999999998 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4906 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6806815129999997, 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037394366999999998, 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.878 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4906 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179303+0000", 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8219999999999998e-06 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179312+0000", 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6419999999999996e-06 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179319+0000", 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6449999999999999e-06 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216272+0000", 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036953142000000001 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216352+0000", 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0044999999999997e-05 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216694+0000", 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034238399999999999 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.882 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5226 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6672851150000003, 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029942795000000001, 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5226 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:11.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192697+0000", 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7199999999999998e-07 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192701+0000", 2024-09-21T23:10:11.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8e-06 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192712+0000", 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1156e-05 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192724+0000", 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2551999999999999e-05 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222110+0000", 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029385620000000001 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222191+0000", 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1520000000000006e-05 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222639+0000", 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044747500000000002 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.886 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5227 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6672457749999996, 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030372671, 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5227 2024-09-21T23:10:11.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192736+0000", 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192742+0000", 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2260000000000002e-06 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192753+0000", 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0762e-05 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192761+0000", 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.14e-06 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222674+0000", 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029912979999999999 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222754+0000", 2024-09-21T23:10:11.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9816999999999998e-05 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223108+0000", 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035405799999999998 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5228 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6672088609999998, 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0320895, 2024-09-21T23:10:11.891 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5228 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192773+0000", 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192776+0000", 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2380000000000002e-06 2024-09-21T23:10:11.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192787+0000", 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0764999999999999e-05 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192795+0000", 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3000000000000002e-06 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223154+0000", 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030358672999999999 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223230+0000", 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6268999999999998e-05 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224862+0000", 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001631635 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5229 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6671746780000003, 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032496191000000001, 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5229 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.896 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:11.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5199999999999996e-07 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192810+0000", 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192820+0000", 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0132000000000001e-05 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192829+0000", 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5900000000000008e-06 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224912+0000", 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032083410999999999 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224985+0000", 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2455999999999999e-05 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225303+0000", 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031788999999999998 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5230 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6671400040000002, 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033015543000000001, 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5230 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:11.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192842+0000", 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192845+0000", 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192856+0000", 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1008e-05 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192865+0000", 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225398+0000", 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032533617000000001 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225489+0000", 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1038999999999994e-05 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225857+0000", 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000367597 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5231 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.904 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6671060820000001, 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033349594000000003, 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5231 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192876+0000", 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192879+0000", 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2059999999999999e-06 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192890+0000", 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0702e-05 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192898+0000", 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3299999999999999e-06 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225903+0000", 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033004622999999997 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225981+0000", 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7990000000000004e-05 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226225+0000", 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024411499999999999 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5232 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3801088~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6670721119999996, 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033795809000000003, 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5232 2024-09-21T23:10:11.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192910+0000", 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1800000000000005e-07 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192913+0000", 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1669999999999999e-06 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192924+0000", 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0692e-05 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192936+0000", 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2414e-05 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226252+0000", 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03331626 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226309+0000", 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6733e-05 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226705+0000", 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00039582499999999999 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5404 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6623969189999999, 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029524524, 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5404 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:11.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197586+0000", 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197593+0000", 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1350000000000003e-06 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198348+0000", 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00075463500000000003 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198360+0000", 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800999999999999e-05 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226750+0000", 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.028390658999999999 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226800+0000", 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0052000000000001e-05 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227109+0000", 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308382 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5406 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6615588639999999, 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029727338999999998, 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5406 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198423+0000", 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7100000000000001e-07 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198427+0000", 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198436+0000", 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198446+0000", 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6360000000000006e-06 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227686+0000", 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029240289999999999 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227764+0000", 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7676000000000006e-05 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228150+0000", 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038574300000000002 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.921 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5407 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6615249859999999, 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030154378999999999, 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5407 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198457+0000", 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198460+0000", 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3409999999999999e-06 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198470+0000", 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198478+0000", 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7130000000000005e-06 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228216+0000", 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029738243000000001 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228290+0000", 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3461999999999996e-05 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228611+0000", 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032108100000000002 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5408 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6614935199999996, 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030920828000000001, 2024-09-21T23:10:11.926 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5408 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198491+0000", 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198501+0000", 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5719999999999992e-06 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198508+0000", 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2470000000000002e-06 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228720+0000", 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030211598999999999 2024-09-21T23:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228851+0000", 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013098299999999999 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229409+0000", 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055781799999999996 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5411 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3440640~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6594367459999999, 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029725732000000001, 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5411 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7100000000000002e-07 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200549+0000", 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200558+0000", 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8400000000000007e-06 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200573+0000", 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5006e-05 2024-09-21T23:10:11.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229813+0000", 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029239580000000001 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229897+0000", 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3572999999999994e-05 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230270+0000", 2024-09-21T23:10:11.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037366400000000001 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5412 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6593885109999995, 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030113958, 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5412 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4099999999999998e-07 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200626+0000", 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2672999999999997e-05 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200637+0000", 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1248e-05 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200646+0000", 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6559999999999998e-06 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230319+0000", 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029672765 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230378+0000", 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8696999999999997e-05 2024-09-21T23:10:11.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230707+0000", 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032917800000000001 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.78529526699999996, 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:11.939 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:11.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:11.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.61952841599999997, 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:11.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:11.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:11.951 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:11.951 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:11.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:11.952 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:11.950+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:12.053 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:12.054+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:12.085 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:12.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:12.085 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:12.085 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:12.098 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:12.154 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:12.150+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:12.255 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:12.250+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:12.264 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:12.264 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:12.264 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:12.265 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:12.265 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:12.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:12.355 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:12.354+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:12.399 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:12.399 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5124 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2127636930000003, 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046086192999999998, 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.400 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5124 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186881+0000", 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.133e-06 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186887+0000", 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7939999999999999e-06 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186904+0000", 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7193e-05 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186918+0000", 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223027+0000", 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036108460000000002 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223201+0000", 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017483900000000001 2024-09-21T23:10:12.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.232966+0000", 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097649039999999996 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5279 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2114850529999996, 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045406334999999999, 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.404 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5279 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188160+0000", 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188167+0000", 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0700000000000001e-06 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188188+0000", 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-05 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188205+0000", 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7444000000000001e-05 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233051+0000", 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044846496999999999 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233167+0000", 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115307 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233565+0000", 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039829399999999999 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5280 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:12.408 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2114158570000004, 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047857909999999997, 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5280 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188229+0000", 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.237e-06 2024-09-21T23:10:12.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188235+0000", 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188254+0000", 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9015999999999999e-05 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188268+0000", 2024-09-21T23:10:12.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.395e-05 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233608+0000", 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045339957 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233722+0000", 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011403700000000001 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236086+0000", 2024-09-21T23:10:12.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00236359 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5281 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2113543570000003, 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048399091999999998, 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.413 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5281 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188291+0000", 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188298+0000", 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188315+0000", 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7819999999999999e-05 2024-09-21T23:10:12.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188329+0000", 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3383e-05 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236130+0000", 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047800827999999997 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236240+0000", 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110745 2024-09-21T23:10:12.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236689+0000", 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044823600000000002 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5282 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2112964780000004, 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048602487, 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.417 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5282 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188348+0000", 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188355+0000", 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3029999999999996e-06 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188373+0000", 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8056000000000001e-05 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188386+0000", 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3536999999999999e-05 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236748+0000", 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048361617000000003 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236840+0000", 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2075000000000002e-05 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236950+0000", 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109755 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.421 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5283 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2111532189999998, 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048844519000000003, 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5283 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188493+0000", 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3800000000000001e-06 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188503+0000", 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188526+0000", 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.304e-05 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188543+0000", 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7557000000000001e-05 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237020+0000", 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048476656 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237187+0000", 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016670199999999999 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237335+0000", 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014857100000000001 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.425 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5284 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2110713530000004, 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049117881000000002, 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5284 2024-09-21T23:10:12.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188574+0000", 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188580+0000", 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188598+0000", 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8304e-05 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188611+0000", 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3329000000000001e-05 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237402+0000", 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048790838000000003 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237549+0000", 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146897 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237690+0000", 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141383 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5285 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2110140669999998, 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049392706000000001, 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.430 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5285 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188631+0000", 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0400000000000005e-07 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188636+0000", 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8329999999999999e-06 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188655+0000", 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8246000000000001e-05 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188670+0000", 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5440000000000001e-05 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237748+0000", 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049077975000000003 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237890+0000", 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141953 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238022+0000", 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013235499999999999 2024-09-21T23:10:12.434 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5286 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2109549639999999, 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049655168, 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.435 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5286 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188690+0000", 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1769999999999999e-06 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188696+0000", 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5269999999999998e-06 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188713+0000", 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7836e-05 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188726+0000", 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238077+0000", 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049350964999999997 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238228+0000", 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015027000000000001 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238344+0000", 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116403 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5287 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2101853760000001, 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049431115999999997, 2024-09-21T23:10:12.439 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5287 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189460+0000", 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189470+0000", 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189491+0000", 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0743000000000001e-05 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189503+0000", 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2787e-05 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238395+0000", 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048891284 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238519+0000", 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124745 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238890+0000", 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037020400000000001 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.443 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5465 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2023893839999999, 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045516793999999999, 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5465 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197256+0000", 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197262+0000", 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197280+0000", 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197294+0000", 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3983e-05 2024-09-21T23:10:12.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242417+0000", 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045122664999999999 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242550+0000", 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013329499999999999 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242771+0000", 2024-09-21T23:10:12.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022120399999999999 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5466 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2023313269999996, 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045852049999999998, 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.448 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5466 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197313+0000", 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197321+0000", 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197338+0000", 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7824000000000001e-05 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197353+0000", 2024-09-21T23:10:12.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.456e-05 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242853+0000", 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045500119999999998 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243010+0000", 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015682599999999999 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243165+0000", 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015463400000000001 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5467 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2022720839999996, 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046123232, 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.453 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5467 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197373+0000", 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197378+0000", 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197397+0000", 2024-09-21T23:10:12.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8793000000000001e-05 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197411+0000", 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3726999999999999e-05 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243249+0000", 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045838320000000002 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243376+0000", 2024-09-21T23:10:12.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012760199999999999 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243495+0000", 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011848100000000001 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5639 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.1994604730000002, 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045343482999999997, 2024-09-21T23:10:12.457 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5639 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200184+0000", 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200190+0000", 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:10:12.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200209+0000", 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8563000000000002e-05 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200223+0000", 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4173e-05 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.244967+0000", 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044743567999999997 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245103+0000", 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013634300000000001 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245527+0000", 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042370199999999998 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5640 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.1993646160000004, 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045524901, 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5640 2024-09-21T23:10:12.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200280+0000", 2024-09-21T23:10:12.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200287+0000", 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200307+0000", 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0129999999999999e-05 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200321+0000", 2024-09-21T23:10:12.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245576+0000", 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045255521 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245701+0000", 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012480100000000001 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245804+0000", 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102933 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5641 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.1993019770000002, 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045750444000000001, 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5641 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200343+0000", 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200351+0000", 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9160000000000001e-06 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200369+0000", 2024-09-21T23:10:12.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8490000000000001e-05 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200383+0000", 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3234000000000001e-05 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245850+0000", 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045467883000000001 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245979+0000", 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128641 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246092+0000", 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113143 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5642 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:12.470 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.1992418740000002, 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045975347999999999, 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5642 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200403+0000", 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200409+0000", 2024-09-21T23:10:12.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6570000000000002e-06 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200427+0000", 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8306000000000001e-05 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200441+0000", 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4042999999999999e-05 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246144+0000", 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045702495000000003 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246277+0000", 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133831 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246377+0000", 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9835999999999998e-05 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.474 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5643 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.1991843480000002, 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046409156, 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5643 2024-09-21T23:10:12.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200460+0000", 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200466+0000", 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200483+0000", 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7776999999999999e-05 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200500+0000", 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6852999999999999e-05 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246582+0000", 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046081471999999998 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246725+0000", 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143184 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246869+0000", 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014361699999999999 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5644 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.199124275, 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046601900000000002, 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.479 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5644 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200521+0000", 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200526+0000", 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-06 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200544+0000", 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7410000000000001e-05 2024-09-21T23:10:12.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200559+0000", 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5435999999999999e-05 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246920+0000", 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046360902000000002 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247055+0000", 2024-09-21T23:10:12.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468200000000001 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247121+0000", 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6712999999999996e-05 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5645 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.1990655490000002, 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.04674565, 2024-09-21T23:10:12.483 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5645 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200579+0000", 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2270000000000001e-06 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200585+0000", 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:12.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200605+0000", 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9675999999999999e-05 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200618+0000", 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509999999999999e-05 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247148+0000", 2024-09-21T23:10:12.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046529451999999999 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247219+0000", 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0833000000000004e-05 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247324+0000", 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105122 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.487 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:12.488 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:12.488 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:12.488 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:10:12.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:12.490 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:12.490+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:12.580 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone8 from 0xd98aad6 (overlap 0x5b7efa) to 0x5258ad 2024-09-21T23:10:12.590 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:12.590+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:12.672 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:12.672 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:12.672 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:12.672 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:12.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:12.727 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-21T23:10:12.729 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:12.730+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:12.792 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:12.790+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:12.851 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:12.851 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:12.851 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:12.852 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:12.852 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:12.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:12.909 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:12.910+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:12.933 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:12.967 INFO:teuthology.orchestra.run.smithi150.stdout:806 read 0x8bb6a05 thru 0x8bbbc1b (0x5217 bytes) 2024-09-21T23:10:12.967 INFO:teuthology.orchestra.run.smithi150.stdout:807 writesame 0x40fb5bc thru 0x4109e9b data_size 0x90(0xe8e0 bytes) 2024-09-21T23:10:12.994 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:12.994+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:12.997 INFO:teuthology.orchestra.run.smithi150.stdout:808 read 0xc7b3177 thru 0xc7bcd12 (0x9b9c bytes) 2024-09-21T23:10:12.998 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:12.998 INFO:teuthology.orchestra.run.smithi150.stdout:811 read 0xceeeda1 thru 0xcefd3d1 (0xe631 bytes) 2024-09-21T23:10:13.015 INFO:teuthology.orchestra.run.smithi150.stdout:812 read 0x61de567 thru 0x61e1527 (0x2fc1 bytes) 2024-09-21T23:10:13.015 INFO:teuthology.orchestra.run.smithi150.stdout:813 read 0xd8f7285 thru 0xd90267f (0xb3fb bytes) 2024-09-21T23:10:13.015 INFO:teuthology.orchestra.run.smithi150.stdout:814 punch from 0x99e27c4 to 0x99e8644, (0x5e80 bytes) 2024-09-21T23:10:13.015 INFO:teuthology.orchestra.run.smithi150.stdout:816 punch from 0x335075 to 0x3428c3, (0xd84e bytes) 2024-09-21T23:10:13.015 INFO:teuthology.orchestra.run.smithi150.stdout:817 read 0x3ac8738 thru 0x3acd352 (0x4c1b bytes) 2024-09-21T23:10:13.015 INFO:teuthology.orchestra.run.smithi150.stdout:818 write 0xd721a70 thru 0xd722b77 (0x1108 bytes) 2024-09-21T23:10:13.035 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3817 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.1447525089999999, 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020519229, 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3817 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892626+0000", 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892631+0000", 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.57e-06 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892648+0000", 2024-09-21T23:10:13.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7323e-05 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892663+0000", 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4610000000000001e-05 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908171+0000", 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015508061 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908384+0000", 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000212945 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.913144+0000", 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047596000000000001 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3818 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 9895936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:13.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.1446956689999999, 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016964216000000001, 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3818 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:13.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892682+0000", 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892688+0000", 2024-09-21T23:10:13.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7230000000000001e-06 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892707+0000", 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8423999999999998e-05 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892720+0000", 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3900000000000001e-05 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908948+0000", 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016227075000000001 2024-09-21T23:10:13.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909223+0000", 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027500799999999998 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909645+0000", 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000422963 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.044 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3821 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 15663104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 6.1446400690000003, 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905422000000001, 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3821 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:13.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892738+0000", 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892744+0000", 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:10:13.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892763+0000", 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8916e-05 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892777+0000", 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4216000000000001e-05 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909722+0000", 2024-09-21T23:10:13.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016944846999999999 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909918+0000", 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019664199999999999 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.910642+0000", 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072407799999999996 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6407 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2687254049999996, 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01587125, 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6407 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768653+0000", 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2729999999999999e-06 2024-09-21T23:10:13.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768658+0000", 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768676+0000", 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8280000000000001e-05 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768689+0000", 2024-09-21T23:10:13.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2823000000000001e-05 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784031+0000", 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015341429 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784270+0000", 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023918799999999999 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784523+0000", 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025289399999999998 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6409 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2686683729999997, 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016872036999999999, 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.053 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6409 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768710+0000", 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.877e-06 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768716+0000", 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768735+0000", 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8983000000000001e-05 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768750+0000", 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4246999999999999e-05 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785004+0000", 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016254497 2024-09-21T23:10:13.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785251+0000", 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024662499999999999 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785581+0000", 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032997799999999998 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6411 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2685549089999997, 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017242586000000001, 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6411 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768824+0000", 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6750000000000001e-06 2024-09-21T23:10:13.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768847+0000", 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3706999999999999e-05 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768878+0000", 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000003e-05 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768899+0000", 2024-09-21T23:10:13.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0468000000000001e-05 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785675+0000", 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016776669000000001 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785818+0000", 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014246199999999999 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786065+0000", 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024697500000000003 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6413 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2684440930000003, 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017559430000000001, 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.062 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6413 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768934+0000", 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4929999999999999e-06 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768948+0000", 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768966+0000", 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8519999999999999e-05 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768980+0000", 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3725e-05 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786147+0000", 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017167094000000001 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786280+0000", 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013344499999999999 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786492+0000", 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021196600000000001 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6415 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2683762180000002, 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905754999999999, 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6415 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769002+0000", 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769018+0000", 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6337999999999998e-05 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769036+0000", 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8042000000000001e-05 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769050+0000", 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786564+0000", 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017514538999999999 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786700+0000", 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786906+0000", 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000206689 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.070 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6418 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.268305711, 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018383022999999998, 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6418 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:13.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769072+0000", 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769078+0000", 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4219999999999999e-06 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769094+0000", 2024-09-21T23:10:13.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5815e-05 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769107+0000", 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3305000000000001e-05 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787013+0000", 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017906090999999999 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787208+0000", 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019526600000000001 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787454+0000", 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024600099999999998 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6420 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:13.075 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2682541819999997, 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018780271000000001, 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6420 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769124+0000", 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3799999999999996e-07 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769129+0000", 2024-09-21T23:10:13.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769146+0000", 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198e-05 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769158+0000", 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1987999999999999e-05 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787568+0000", 2024-09-21T23:10:13.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018410037000000001 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787697+0000", 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012926600000000001 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787903+0000", 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205692 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6421 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2682032919999999, 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019118928, 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6421 2024-09-21T23:10:13.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769179+0000", 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.78e-06 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769194+0000", 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4576000000000001e-05 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769204+0000", 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0516999999999999e-05 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787991+0000", 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018786981000000001 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788130+0000", 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013833899999999999 2024-09-21T23:10:13.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788293+0000", 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000162905 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6423 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2681566149999997, 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019463588, 2024-09-21T23:10:13.084 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6423 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769221+0000", 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999998e-07 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769226+0000", 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8799999999999999e-06 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769235+0000", 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8300000000000002e-06 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769241+0000", 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2580000000000001e-06 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788361+0000", 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019119873999999999 2024-09-21T23:10:13.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788514+0000", 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152989 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788684+0000", 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016989899999999999 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6426 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2681271870000002, 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019969425999999998, 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6426 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:13.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769253+0000", 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5490000000000001e-06 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769260+0000", 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5650000000000004e-06 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769270+0000", 2024-09-21T23:10:13.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9180000000000006e-06 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788787+0000", 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019516977000000001 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788981+0000", 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019366300000000001 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789219+0000", 2024-09-21T23:10:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023821399999999999 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6427 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2680948179999998, 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020632774999999999, 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6427 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769283+0000", 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1480000000000001e-06 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769288+0000", 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3819999999999997e-06 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769301+0000", 2024-09-21T23:10:13.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2968e-05 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769309+0000", 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1040000000000007e-06 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789376+0000", 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020067406999999999 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789621+0000", 2024-09-21T23:10:13.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024461200000000001 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789915+0000", 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029415399999999998 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6429 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.268005144, 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021034428000000001, 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.097 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6429 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769374+0000", 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.906e-06 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769385+0000", 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1707e-05 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769410+0000", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4102999999999999e-05 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769429+0000", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9110000000000002e-05 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790009+0000", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020580751000000001 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790186+0000", 2024-09-21T23:10:13.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000176708 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790406+0000", 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022014300000000001 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6431 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2679170019999999, 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021448724999999998, 2024-09-21T23:10:13.101 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6431 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769461+0000", 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.403e-06 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769468+0000", 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4069999999999998e-06 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769487+0000", 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769501+0000", 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4323e-05 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790548+0000", 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021047204 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790744+0000", 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019580199999999999 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790909+0000", 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164709 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6433 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.105 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2678570520000001, 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021886459, 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6433 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769521+0000", 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769527+0000", 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7100000000000004e-06 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769544+0000", 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7229999999999999e-05 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769558+0000", 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3567000000000001e-05 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790998+0000", 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021440516999999999 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791177+0000", 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017838899999999999 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791406+0000", 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022983300000000001 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.109 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6436 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2678021590000004, 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022294958, 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6436 2024-09-21T23:10:13.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769576+0000", 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769594+0000", 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8136999999999999e-05 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769612+0000", 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8063e-05 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769626+0000", 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3706999999999999e-05 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791524+0000", 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021898407000000002 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791653+0000", 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129116 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791870+0000", 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021640200000000001 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6437 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2677080759999999, 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022700130999999998, 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.114 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6437 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769671+0000", 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6530000000000001e-06 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769680+0000", 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8830000000000001e-06 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769701+0000", 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0883e-05 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769718+0000", 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6385999999999999e-05 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791966+0000", 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022248622999999999 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792154+0000", 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000187834 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792369+0000", 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021486899999999999 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6440 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:13.118 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.267638764, 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.023189891000000001, 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6440 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:13.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769740+0000", 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.327e-06 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769745+0000", 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6760000000000004e-06 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769762+0000", 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7187e-05 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769776+0000", 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3233e-05 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:13.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792416+0000", 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022640389 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792608+0000", 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019183899999999999 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792928+0000", 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032024000000000001 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:13.123 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:13.123 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:13.124 INFO:teuthology.orchestra.run.smithi150.stdout:819 writesame 0xec27292 thru 0xec2a13d data_size 0x4(0x2eac bytes) 2024-09-21T23:10:13.124 INFO:teuthology.orchestra.run.smithi150.stdout:820 writesame 0x162d457 thru 0x1635507 data_size 0xb(0x80b1 bytes) 2024-09-21T23:10:13.124 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:13.122+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:13.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:13.137 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:13.157 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:13.157 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:13.179 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:13.179 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:13.198 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:13.198 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:13.198 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:13.220 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:13.220 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:13.224 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:13.226+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:13.254 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:13.254 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:13.289 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:13.289 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:13.293 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:13.293 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:13.300 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:13.300 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:13.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:13.335 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:13.334+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:13.353 INFO:teuthology.orchestra.run.smithi150.stdout:821 punch from 0x4fae3a to 0x5082ea, (0xd4b0 bytes) 2024-09-21T23:10:13.354 INFO:teuthology.orchestra.run.smithi150.stdout:822 writesame 0x9f15fe7 thru 0x9f24806 data_size 0x30(0xe820 bytes) 2024-09-21T23:10:13.426 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:13.426+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:13.427 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:13.427 INFO:teuthology.orchestra.run.smithi150.stdout:825 punch from 0xbfd205b to 0xbfd7923, (0x58c8 bytes) 2024-09-21T23:10:13.427 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:13.427 INFO:teuthology.orchestra.run.smithi150.stdout:828 write 0xb418f98 thru 0xb41d19f (0x4208 bytes) 2024-09-21T23:10:13.489 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:13.489 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:13.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:13.527 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:13.526+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:13.628 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:13.626+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:13.676 INFO:teuthology.orchestra.run.smithi150.stdout:829 trunc from 0xec2a13e to 0x3517a6 2024-09-21T23:10:13.676 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:13.676 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:13.696 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:13.739 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:13.738+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:13.829 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:13.826+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:13.857 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:13.857 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:13.876 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:13.929 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:13.930+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:14.030 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:14.026+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:14.068 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:14.068 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:14.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:14.116 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:14.130 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:14.130+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:14.133 INFO:teuthology.orchestra.run.smithi150.stdout:832 write 0x586f037 thru 0x587627b (0x7245 bytes) 2024-09-21T23:10:14.152 INFO:teuthology.orchestra.run.smithi150.stdout:834 punch from 0xbf69e to 0xc9655, (0x9fb7 bytes) 2024-09-21T23:10:14.152 INFO:teuthology.orchestra.run.smithi150.stdout:835 write 0xda53bc thru 0xda695a (0x159f bytes) 2024-09-21T23:10:14.163 INFO:teuthology.orchestra.run.smithi150.stdout:839 read 0x5840829 thru 0x5847723 (0x6efb bytes) 2024-09-21T23:10:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:840 punch from 0x3ca1fff to 0x3cafbb1, (0xdbb2 bytes) 2024-09-21T23:10:14.165 INFO:teuthology.orchestra.run.smithi150.stdout:841 read 0x3d93625 thru 0x3da3535 (0xff11 bytes) 2024-09-21T23:10:14.177 INFO:teuthology.orchestra.run.smithi150.stdout:842 write 0x8533064 thru 0x8536dba (0x3d57 bytes) 2024-09-21T23:10:14.197 INFO:teuthology.orchestra.run.smithi150.stdout:843 punch from 0x843a03c to 0x843b5fa, (0x15be bytes) 2024-09-21T23:10:14.197 INFO:teuthology.orchestra.run.smithi150.stdout:844 read 0x4ebc61a thru 0x4eca113 (0xdafa bytes) 2024-09-21T23:10:14.210 INFO:teuthology.orchestra.run.smithi150.stdout:845 write 0xa0ecfba thru 0xa0f7b69 (0xabb0 bytes) 2024-09-21T23:10:14.229 INFO:teuthology.orchestra.run.smithi150.stdout:846 punch from 0xaa648c to 0xab293a, (0xc4ae bytes) 2024-09-21T23:10:14.231 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:14.230+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:14.236 INFO:teuthology.orchestra.run.smithi150.stdout:847 trunc from 0xa0f7b6a to 0x5c6d50c 2024-09-21T23:10:14.269 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T23:10:14.269 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-21T23:10:14.269 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:10:14.269 DEBUG:teuthology.orchestra.run.smithi175:> type ceph-objectstore-tool 2024-09-21T23:10:14.270 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:14.270 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:14.275 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:10:14.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-21T23:10:14.278 INFO:teuthology.orchestra.run.smithi150.stdout:849 write 0x93b75ff thru 0x93bca5c (0x545e bytes) 2024-09-21T23:10:14.287 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:14.297 INFO:teuthology.orchestra.run.smithi150.stdout:851 writesame 0x292d609 thru 0x293cd7e data_size 0x46(0xf776 bytes) 2024-09-21T23:10:14.318 INFO:teuthology.orchestra.run.smithi150.stdout:852 read 0x1132543 thru 0x113999b (0x7459 bytes) 2024-09-21T23:10:14.318 INFO:teuthology.orchestra.run.smithi150.stdout:853 write 0x8e0c83e thru 0x8e11eef (0x56b2 bytes) 2024-09-21T23:10:14.332 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:14.330+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:14.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-21T23:10:14.348 INFO:teuthology.orchestra.run.smithi150.stdout:854 write 0xc4f55f0 thru 0xc4faf74 (0x5985 bytes) 2024-09-21T23:10:14.366 INFO:teuthology.orchestra.run.smithi150.stdout:855 read 0x4c42ebe thru 0x4c4bf4a (0x908d bytes) 2024-09-21T23:10:14.366 INFO:teuthology.orchestra.run.smithi150.stdout:856 punch from 0x34443a2 to 0x344f274, (0xaed2 bytes) 2024-09-21T23:10:14.366 INFO:teuthology.orchestra.run.smithi150.stdout:858 write 0x23cdb6e thru 0x23d9ced (0xc180 bytes) 2024-09-21T23:10:14.383 INFO:teuthology.orchestra.run.smithi150.stdout:860 punch from 0x48079df to 0x480c209, (0x482a bytes) 2024-09-21T23:10:14.396 INFO:teuthology.orchestra.run.smithi150.stdout:861 trunc from 0xc4faf75 to 0x32dbf19 2024-09-21T23:10:14.433 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:14.430+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:14.444 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:14.444 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:14.452 INFO:teuthology.orchestra.run.smithi150.stdout:863 writesame 0xd8cdd06 thru 0xd8ceb2a data_size 0x3(0xe25 bytes) 2024-09-21T23:10:14.452 INFO:teuthology.orchestra.run.smithi150.stdout:865 punch from 0x33778fa to 0x3385992, (0xe098 bytes) 2024-09-21T23:10:14.452 INFO:teuthology.orchestra.run.smithi150.stdout:867 write 0x30094df thru 0x3014d97 (0xb8b9 bytes) 2024-09-21T23:10:14.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:14.472 INFO:teuthology.orchestra.run.smithi150.stdout:869 punch from 0x5ad7bca to 0x5ae1a7f, (0x9eb5 bytes) 2024-09-21T23:10:14.472 INFO:teuthology.orchestra.run.smithi150.stdout:870 read 0x5944437 thru 0x595106d (0xcc37 bytes) 2024-09-21T23:10:14.472 INFO:teuthology.orchestra.run.smithi150.stdout:872 write 0xbf4902e thru 0xbf4e4d8 (0x54ab bytes) 2024-09-21T23:10:14.490 INFO:teuthology.orchestra.run.smithi150.stdout:873 write 0x477276 thru 0x483e3d (0xcbc8 bytes) 2024-09-21T23:10:14.503 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:10:14.519 INFO:teuthology.orchestra.run.smithi150.stdout:874 write 0xe254adf thru 0xe263b7e (0xf0a0 bytes) 2024-09-21T23:10:14.533 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:14.534+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:14.544 INFO:teuthology.orchestra.run.smithi150.stdout:875 write 0x930db27 thru 0x931169d (0x3b77 bytes) 2024-09-21T23:10:14.564 INFO:teuthology.orchestra.run.smithi150.stdout:877 writesame 0x8cdb85f thru 0x8cdbdad data_size 0x3(0x54f bytes) 2024-09-21T23:10:14.569 INFO:teuthology.orchestra.run.smithi150.stdout:883 read 0x5cd1fd thru 0x5d679b (0x959f bytes) 2024-09-21T23:10:14.571 INFO:teuthology.orchestra.run.smithi150.stdout:884 write 0x67bbaef thru 0x67bfe04 (0x4316 bytes) 2024-09-21T23:10:14.590 INFO:teuthology.orchestra.run.smithi150.stdout:885 trunc from 0xe263b7f to 0xd34574d 2024-09-21T23:10:14.609 INFO:teuthology.orchestra.run.smithi150.stdout:886 write 0x3b7d9fd thru 0x3b8c256 (0xe85a bytes) 2024-09-21T23:10:14.629 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:14.629 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:14.634 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:14.629+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:14.640 INFO:teuthology.orchestra.run.smithi150.stdout:887 read 0x634302b thru 0x63476be (0x4694 bytes) 2024-09-21T23:10:14.640 INFO:teuthology.orchestra.run.smithi150.stdout:890 punch from 0x5a2f775 to 0x5a3ed16, (0xf5a1 bytes) 2024-09-21T23:10:14.640 INFO:teuthology.orchestra.run.smithi150.stdout:891 read 0x8f64505 thru 0x8f6ade8 (0x68e4 bytes) 2024-09-21T23:10:14.641 INFO:teuthology.orchestra.run.smithi150.stdout:892 read 0x62850da thru 0x628acfe (0x5c25 bytes) 2024-09-21T23:10:14.647 INFO:teuthology.orchestra.run.smithi150.stdout:893 trunc from 0xd34574d to 0x96ce1d3 2024-09-21T23:10:14.656 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:14.668 INFO:teuthology.orchestra.run.smithi150.stdout:894 punch from 0x5f7e313 to 0x5f85e24, (0x7b11 bytes) 2024-09-21T23:10:14.668 INFO:teuthology.orchestra.run.smithi150.stdout:895 write 0x97dd55 thru 0x98623e (0x84ea bytes) 2024-09-21T23:10:14.686 INFO:teuthology.orchestra.run.smithi150.stdout:897 write 0x17b8ff6 thru 0x17c425d (0xb268 bytes) 2024-09-21T23:10:14.704 INFO:teuthology.orchestra.run.smithi150.stdout:898 write 0x7756523 thru 0x77567db (0x2b9 bytes) 2024-09-21T23:10:14.708 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:14.708 INFO:teuthology.orchestra.run.smithi150.stdout:900 punch from 0x2356a0 to 0x23a98a, (0x52ea bytes) 2024-09-21T23:10:14.708 INFO:teuthology.orchestra.run.smithi150.stdout:901 write 0x217536c thru 0x2175f4c (0xbe1 bytes) 2024-09-21T23:10:14.715 INFO:teuthology.orchestra.run.smithi150.stdout:902 punch from 0x1c0e70a to 0x1c1cf21, (0xe817 bytes) 2024-09-21T23:10:14.715 INFO:teuthology.orchestra.run.smithi150.stdout:903 write 0x858726 thru 0x85bfe6 (0x38c1 bytes) 2024-09-21T23:10:14.720 INFO:teuthology.orchestra.run.smithi150.stdout:904 write 0x3b15935 thru 0x3b16505 (0xbd1 bytes) 2024-09-21T23:10:14.725 INFO:teuthology.orchestra.run.smithi150.stdout:908 trunc from 0x96ce1d3 to 0x86f7f3a 2024-09-21T23:10:14.736 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:14.734+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:14.746 INFO:teuthology.orchestra.run.smithi150.stdout:909 read 0x837a69 thru 0x83a7bf (0x2d57 bytes) 2024-09-21T23:10:14.747 INFO:teuthology.orchestra.run.smithi150.stdout:911 write 0x2171151 thru 0x217a215 (0x90c5 bytes) 2024-09-21T23:10:14.761 INFO:teuthology.orchestra.run.smithi150.stdout:912 punch from 0x6f13ac to 0x6fcd72, (0xb9c6 bytes) 2024-09-21T23:10:14.761 INFO:teuthology.orchestra.run.smithi150.stdout:913 read 0x3883908 thru 0x38882a8 (0x49a1 bytes) 2024-09-21T23:10:14.762 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:14.762 INFO:teuthology.orchestra.run.smithi150.stdout:917 punch from 0x598f305 to 0x599c576, (0xd271 bytes) 2024-09-21T23:10:14.762 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:14.781 INFO:teuthology.orchestra.run.smithi150.stdout:920 writesame 0xddc47d4 thru 0xddceb31 data_size 0xb(0xa35e bytes) 2024-09-21T23:10:14.781 INFO:teuthology.orchestra.run.smithi150.stdout:921 read 0x111ae75 thru 0x11277ce (0xc95a bytes) 2024-09-21T23:10:14.781 INFO:teuthology.orchestra.run.smithi150.stdout:922 writesame 0x84ff70 thru 0x85f93b data_size 0x6(0xf9cc bytes) 2024-09-21T23:10:14.790 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:14.790 INFO:teuthology.orchestra.run.smithi150.stdout:924 read 0x98e9607 thru 0x98eee2b (0x5825 bytes) 2024-09-21T23:10:14.801 INFO:teuthology.orchestra.run.smithi150.stdout:925 writesame 0xe90a893 thru 0xe912fd1 data_size 0x3(0x873f bytes) 2024-09-21T23:10:14.802 INFO:teuthology.orchestra.run.smithi150.stdout:927 trunc from 0xe912fd2 to 0xde45673 2024-09-21T23:10:14.810 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:14.810 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:14.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:14.826 INFO:teuthology.orchestra.run.smithi150.stdout:928 write 0xe1aea52 thru 0xe1b4ead (0x645c bytes) 2024-09-21T23:10:14.836 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:14.834+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:14.864 INFO:teuthology.orchestra.run.smithi150.stdout:929 trunc from 0xe1b4eae to 0x1754ffd 2024-09-21T23:10:14.937 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:14.933+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:14.946 INFO:teuthology.orchestra.run.smithi150.stdout:930 write 0x94bccec thru 0x94c49d4 (0x7ce9 bytes) 2024-09-21T23:10:14.965 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:14.965 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:14.970 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:14.970 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:14.978 INFO:teuthology.orchestra.run.smithi150.stdout:935 write 0xc81c983 thru 0xc829572 (0xcbf0 bytes) 2024-09-21T23:10:14.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:14.997 INFO:teuthology.orchestra.run.smithi150.stdout:937 read 0xaa58e78 thru 0xaa5c410 (0x3599 bytes) 2024-09-21T23:10:14.997 INFO:teuthology.orchestra.run.smithi150.stdout:938 write 0x9d2cc8e thru 0x9d337f5 (0x6b68 bytes) 2024-09-21T23:10:15.026 INFO:teuthology.orchestra.run.smithi150.stdout:939 trunc from 0xc829573 to 0x53067d6 2024-09-21T23:10:15.038 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:15.033+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:940 punch from 0x22538f1 to 0x22616d6, (0xdde5 bytes) 2024-09-21T23:10:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:941 read 0xb2b0d9 thru 0xb2dabb (0x29e3 bytes) 2024-09-21T23:10:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:942 punch from 0x41518df to 0x4152459, (0xb7a bytes) 2024-09-21T23:10:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:943 punch from 0x10125ca to 0x101b8e9, (0x931f bytes) 2024-09-21T23:10:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:944 trunc from 0x53067d6 to 0x5befa71 2024-09-21T23:10:15.051 INFO:teuthology.orchestra.run.smithi150.stdout:946 read 0x42f497c thru 0x42fe6c8 (0x9d4d bytes) 2024-09-21T23:10:15.063 INFO:teuthology.orchestra.run.smithi150.stdout:949 trunc from 0x5befa71 to 0xdf5e106 2024-09-21T23:10:15.070 INFO:teuthology.orchestra.run.smithi150.stdout:951 clone 10 order 24 su 262144 sc 3 2024-09-21T23:10:15.125 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:15.125 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:15.139 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:15.140 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:15.137+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:15.240 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:15.238+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:15.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:15.305 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:15.305 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:15.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:15.363 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:15.361+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:15.431 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:15.431 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:15.442 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:15.437+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:15.448 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:15.448 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:15.465 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:15.465 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:15.482 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:15.482 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:15.482 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:15.492 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:15.492 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:15.492 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:15.492 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:15.502 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:15.503 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:15.506 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:15.523 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:15.524 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:15.544 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:15.544 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:15.545 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:15.541+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:15.549 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:10:15.549 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:15.550 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:15.644 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:15.641+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:15.648 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:15.648 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:15.648 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:15.648 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:15.648 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:15.656 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone9 from 0xdf5e106 (overlap 0x3517a6) to 0x54f2a 2024-09-21T23:10:15.665 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:15.755 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:15.753+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:15.795 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-21T23:10:15.809 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:15.809 INFO:teuthology.orchestra.run.smithi150.stdout:953 writesame 0x111f980 thru 0x1127b12 data_size 0x1(0x8193 bytes) 2024-09-21T23:10:15.843 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:15.843 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:15.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4902 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6639221460000009, 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030793945, 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4902 2024-09-21T23:10:15.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.37e-07 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179183+0000", 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9280000000000002e-06 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179192+0000", 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7020000000000007e-06 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179198+0000", 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6780000000000003e-06 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201407+0000", 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022208301999999999 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201500+0000", 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3231000000000002e-05 2024-09-21T23:10:15.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.209974+0000", 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084735669999999996 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4903 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2359296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6638941710000008, 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031240032000000001, 2024-09-21T23:10:15.848 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4903 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179211+0000", 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6630000000000002e-06 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179220+0000", 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8480000000000007e-06 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179226+0000", 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2650000000000002e-06 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210016+0000", 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030790128 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210066+0000", 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9713000000000003e-05 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210448+0000", 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038190499999999998 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.852 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4904 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6638669440000005, 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031636297000000001, 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4904 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179238+0000", 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5349999999999999e-06 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179247+0000", 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6449999999999994e-06 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179260+0000", 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3431999999999999e-05 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210480+0000", 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031220121 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210550+0000", 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0352000000000007e-05 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210871+0000", 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000320701 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.856 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4905 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6638317790000006, 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036964023999999998, 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4905 2024-09-21T23:10:15.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179271+0000", 2024-09-21T23:10:15.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179274+0000", 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1350000000000001e-06 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179283+0000", 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1779999999999992e-06 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179290+0000", 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210903+0000", 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031613162 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210976+0000", 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2591999999999999e-05 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216234+0000", 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052583309999999998 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4906 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6638020359999999, 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037394366999999998, 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4906 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179303+0000", 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8219999999999998e-06 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179312+0000", 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6419999999999996e-06 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179319+0000", 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6449999999999999e-06 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216272+0000", 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036953142000000001 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216352+0000", 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0044999999999997e-05 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216694+0000", 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034238399999999999 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5226 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6504056380000005, 2024-09-21T23:10:15.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029942795000000001, 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5226 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192697+0000", 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7199999999999998e-07 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192701+0000", 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8e-06 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192712+0000", 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1156e-05 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192724+0000", 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2551999999999999e-05 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222110+0000", 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029385620000000001 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222191+0000", 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1520000000000006e-05 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222639+0000", 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044747500000000002 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5227 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6503662979999998, 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030372671, 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5227 2024-09-21T23:10:15.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192736+0000", 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192742+0000", 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2260000000000002e-06 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192753+0000", 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0762e-05 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192761+0000", 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.14e-06 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222674+0000", 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029912979999999999 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222754+0000", 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9816999999999998e-05 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223108+0000", 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035405799999999998 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5228 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6503293839999991, 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0320895, 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.874 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5228 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192773+0000", 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192776+0000", 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2380000000000002e-06 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192787+0000", 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0764999999999999e-05 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192795+0000", 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3000000000000002e-06 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223154+0000", 2024-09-21T23:10:15.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030358672999999999 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223230+0000", 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6268999999999998e-05 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224862+0000", 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001631635 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5229 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6502952010000005, 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032496191000000001, 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5229 2024-09-21T23:10:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5199999999999996e-07 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192810+0000", 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192820+0000", 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0132000000000001e-05 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192829+0000", 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5900000000000008e-06 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224912+0000", 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032083410999999999 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224985+0000", 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2455999999999999e-05 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225303+0000", 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031788999999999998 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5230 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6502605270000004, 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033015543000000001, 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.883 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5230 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192842+0000", 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192845+0000", 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192856+0000", 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1008e-05 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192865+0000", 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225398+0000", 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032533617000000001 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225489+0000", 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1038999999999994e-05 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225857+0000", 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000367597 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.887 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5231 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6502266050000003, 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033349594000000003, 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5231 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192876+0000", 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:10:15.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192879+0000", 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2059999999999999e-06 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192890+0000", 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0702e-05 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192898+0000", 2024-09-21T23:10:15.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3299999999999999e-06 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225903+0000", 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033004622999999997 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225981+0000", 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7990000000000004e-05 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226225+0000", 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024411499999999999 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5232 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3801088~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6501926349999998, 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033795809000000003, 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5232 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:15.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192910+0000", 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1800000000000005e-07 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192913+0000", 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1669999999999999e-06 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192924+0000", 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0692e-05 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192936+0000", 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2414e-05 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226252+0000", 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03331626 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226309+0000", 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6733e-05 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226705+0000", 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00039582499999999999 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5404 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.896 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6455174419999992, 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029524524, 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5404 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:15.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197584+0000", 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197586+0000", 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.197593+0000", 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1350000000000003e-06 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198348+0000", 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00075463500000000003 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198360+0000", 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800999999999999e-05 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226750+0000", 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.028390658999999999 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226800+0000", 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0052000000000001e-05 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227109+0000", 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000308382 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5406 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.644679387, 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029727338999999998, 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5406 2024-09-21T23:10:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198423+0000", 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7100000000000001e-07 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198427+0000", 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198436+0000", 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198446+0000", 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6360000000000006e-06 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227686+0000", 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029240289999999999 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227764+0000", 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7676000000000006e-05 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228150+0000", 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038574300000000002 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5407 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6446455090000001, 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030154378999999999, 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5407 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198457+0000", 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198460+0000", 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3409999999999999e-06 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198470+0000", 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:15.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198478+0000", 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7130000000000005e-06 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228216+0000", 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029738243000000001 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228290+0000", 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3461999999999996e-05 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228611+0000", 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032108100000000002 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5408 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6446140430000007, 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030920828000000001, 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5408 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:15.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198491+0000", 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198501+0000", 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5719999999999992e-06 2024-09-21T23:10:15.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198508+0000", 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2470000000000002e-06 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228720+0000", 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030211598999999999 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228851+0000", 2024-09-21T23:10:15.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013098299999999999 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229409+0000", 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055781799999999996 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5411 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3440640~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6425572689999992, 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029725732000000001, 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5411 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200545+0000", 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7100000000000002e-07 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200549+0000", 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200558+0000", 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8400000000000007e-06 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200573+0000", 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5006e-05 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229813+0000", 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029239580000000001 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229897+0000", 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3572999999999994e-05 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230270+0000", 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037366400000000001 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5412 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.6425090339999997, 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030113958, 2024-09-21T23:10:15.917 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5412 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4099999999999998e-07 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200626+0000", 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2672999999999997e-05 2024-09-21T23:10:15.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200637+0000", 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1248e-05 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200646+0000", 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6559999999999998e-06 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230319+0000", 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029672765 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230378+0000", 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8696999999999997e-05 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230707+0000", 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032917800000000001 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:15.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.7684157899999997, 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:15.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6026489389999998, 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:15.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:15.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:15.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:15.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:15.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:15.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:15.933 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:15.934 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:15.935 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:15.929+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:15.958 INFO:teuthology.orchestra.run.smithi150.stdout:954 writesame 0xf4de15 thru 0xf57fa7 data_size 0x13(0xa193 bytes) 2024-09-21T23:10:16.035 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:16.034+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:16.077 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:16.078 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:16.078 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:16.078 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:16.089 INFO:teuthology.orchestra.run.smithi150.stdout:955 trunc from 0xdf5e106 to 0xbfb06f5 2024-09-21T23:10:16.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:16.135 INFO:teuthology.orchestra.run.smithi150.stdout:956 read 0x42b1e60 thru 0x42bdeb3 (0xc054 bytes) 2024-09-21T23:10:16.135 INFO:teuthology.orchestra.run.smithi150.stdout:958 write 0x472e6fd thru 0x473b0e9 (0xc9ed bytes) 2024-09-21T23:10:16.137 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:16.138+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:16.159 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:16.173 INFO:teuthology.orchestra.run.smithi150.stdout:961 trunc from 0xbfb06f5 to 0x705d637 2024-09-21T23:10:16.237 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:16.238+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:16.255 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:16.255 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:16.255 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:16.255 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:16.255 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:16.259 INFO:teuthology.orchestra.run.smithi150.stdout:962 trunc from 0x705d637 to 0xe2eecc8 2024-09-21T23:10:16.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:16.283 INFO:teuthology.orchestra.run.smithi150.stdout:963 trunc from 0xe2eecc8 to 0xa343920 2024-09-21T23:10:16.313 INFO:teuthology.orchestra.run.smithi150.stdout:965 write 0xc32d02f thru 0xc339571 (0xc543 bytes) 2024-09-21T23:10:16.321 INFO:teuthology.orchestra.run.smithi150.stdout:967 write 0x2cd6a39 thru 0x2ce04ab (0x9a73 bytes) 2024-09-21T23:10:16.324 INFO:teuthology.orchestra.run.smithi150.stdout:968 write 0x177502e thru 0x1775608 (0x5db bytes) 2024-09-21T23:10:16.357 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:16.358+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:16.430 INFO:teuthology.orchestra.run.smithi175.stdout:3.f 2024-09-21T23:10:16.430 INFO:teuthology.orchestra.run.smithi175.stdout:3.e 2024-09-21T23:10:16.430 INFO:teuthology.orchestra.run.smithi175.stdout:3.9 2024-09-21T23:10:16.430 INFO:teuthology.orchestra.run.smithi175.stdout:3.3 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.1d 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.1c 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.1a 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.1 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.a 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.1b 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.14 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.13 2024-09-21T23:10:16.431 INFO:teuthology.orchestra.run.smithi175.stdout:3.11 2024-09-21T23:10:16.440 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:16.440 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:16.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:16.440 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:16.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.440 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5124 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.440 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2558273369999995, 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046086192999999998, 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5124 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186881+0000", 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.133e-06 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186887+0000", 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7939999999999999e-06 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186904+0000", 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7193e-05 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186918+0000", 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T23:10:16.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223027+0000", 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036108460000000002 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223201+0000", 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017483900000000001 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.232966+0000", 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097649039999999996 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.445 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5279 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2545486970000006, 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045406334999999999, 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5279 2024-09-21T23:10:16.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188160+0000", 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188167+0000", 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0700000000000001e-06 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188188+0000", 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-05 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188205+0000", 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7444000000000001e-05 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233051+0000", 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044846496999999999 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233167+0000", 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115307 2024-09-21T23:10:16.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233565+0000", 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039829399999999999 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5280 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2544795010000005, 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047857909999999997, 2024-09-21T23:10:16.450 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5280 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188229+0000", 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.237e-06 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188235+0000", 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188254+0000", 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9015999999999999e-05 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188268+0000", 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.395e-05 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233608+0000", 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045339957 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233722+0000", 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011403700000000001 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236086+0000", 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00236359 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5281 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2544180009999994, 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048399091999999998, 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5281 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.455 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188291+0000", 2024-09-21T23:10:16.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188298+0000", 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188315+0000", 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7819999999999999e-05 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188329+0000", 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3383e-05 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236130+0000", 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047800827999999997 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236240+0000", 2024-09-21T23:10:16.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110745 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236689+0000", 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044823600000000002 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5282 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:16.459 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2543601219999996, 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048602487, 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5282 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:16.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188348+0000", 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188355+0000", 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3029999999999996e-06 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188373+0000", 2024-09-21T23:10:16.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8056000000000001e-05 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188386+0000", 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3536999999999999e-05 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236748+0000", 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048361617000000003 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236840+0000", 2024-09-21T23:10:16.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2075000000000002e-05 2024-09-21T23:10:16.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236950+0000", 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109755 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5283 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2542168629999999, 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048844519000000003, 2024-09-21T23:10:16.465 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5283 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188493+0000", 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3800000000000001e-06 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188503+0000", 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188526+0000", 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.304e-05 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188543+0000", 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7557000000000001e-05 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237020+0000", 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048476656 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237187+0000", 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016670199999999999 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237335+0000", 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014857100000000001 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.469 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5284 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2541349969999995, 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049117881000000002, 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5284 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188574+0000", 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188580+0000", 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188598+0000", 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8304e-05 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188611+0000", 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3329000000000001e-05 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237402+0000", 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048790838000000003 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237549+0000", 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146897 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237690+0000", 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141383 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5285 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2540777110000008, 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049392706000000001, 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5285 2024-09-21T23:10:16.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188631+0000", 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0400000000000005e-07 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188636+0000", 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8329999999999999e-06 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188655+0000", 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8246000000000001e-05 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188670+0000", 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5440000000000001e-05 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237748+0000", 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049077975000000003 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237890+0000", 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141953 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238022+0000", 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013235499999999999 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5286 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2540186079999991, 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049655168, 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5286 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188690+0000", 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1769999999999999e-06 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188696+0000", 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5269999999999998e-06 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188713+0000", 2024-09-21T23:10:16.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7836e-05 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188726+0000", 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238077+0000", 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049350964999999997 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238228+0000", 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015027000000000001 2024-09-21T23:10:16.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238344+0000", 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116403 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5287 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2532490200000002, 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049431115999999997, 2024-09-21T23:10:16.482 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5287 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189460+0000", 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189470+0000", 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189491+0000", 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0743000000000001e-05 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189503+0000", 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2787e-05 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238395+0000", 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048891284 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238519+0000", 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124745 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238890+0000", 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037020400000000001 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5465 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.245453028, 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045516793999999999, 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5465 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197256+0000", 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197262+0000", 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197280+0000", 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197294+0000", 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3983e-05 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242417+0000", 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045122664999999999 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242550+0000", 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013329499999999999 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242771+0000", 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022120399999999999 2024-09-21T23:10:16.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5466 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2453949709999996, 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045852049999999998, 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5466 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197313+0000", 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197321+0000", 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197338+0000", 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7824000000000001e-05 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197353+0000", 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.456e-05 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242853+0000", 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045500119999999998 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243010+0000", 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015682599999999999 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243165+0000", 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015463400000000001 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5467 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2453357280000006, 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046123232, 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.495 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5467 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:16.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197373+0000", 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197378+0000", 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:10:16.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.498 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:16.438+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:16.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197397+0000", 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8793000000000001e-05 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197411+0000", 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3726999999999999e-05 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243249+0000", 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045838320000000002 2024-09-21T23:10:16.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243376+0000", 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012760199999999999 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243495+0000", 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011848100000000001 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5639 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2425241170000003, 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045343482999999997, 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5639 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200184+0000", 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T23:10:16.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200190+0000", 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200209+0000", 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8563000000000002e-05 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200223+0000", 2024-09-21T23:10:16.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4173e-05 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.244967+0000", 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044743567999999997 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245103+0000", 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013634300000000001 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245527+0000", 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042370199999999998 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5640 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2424282600000005, 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045524901, 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.505 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5640 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200280+0000", 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200287+0000", 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200307+0000", 2024-09-21T23:10:16.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0129999999999999e-05 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200321+0000", 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245576+0000", 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045255521 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245701+0000", 2024-09-21T23:10:16.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012480100000000001 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245804+0000", 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102933 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5641 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2423656209999994, 2024-09-21T23:10:16.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045750444000000001, 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5641 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200343+0000", 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200351+0000", 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9160000000000001e-06 2024-09-21T23:10:16.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200369+0000", 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8490000000000001e-05 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200383+0000", 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3234000000000001e-05 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245850+0000", 2024-09-21T23:10:16.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045467883000000001 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245979+0000", 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128641 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246092+0000", 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113143 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5642 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2423055180000002, 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045975347999999999, 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5642 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.514 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200403+0000", 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200409+0000", 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6570000000000002e-06 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200427+0000", 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8306000000000001e-05 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200441+0000", 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4042999999999999e-05 2024-09-21T23:10:16.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246144+0000", 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045702495000000003 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246277+0000", 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133831 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246377+0000", 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9835999999999998e-05 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5643 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2422479919999994, 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046409156, 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.518 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5643 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200460+0000", 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200466+0000", 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200483+0000", 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7776999999999999e-05 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200500+0000", 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6852999999999999e-05 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246582+0000", 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046081471999999998 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246725+0000", 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143184 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246869+0000", 2024-09-21T23:10:16.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014361699999999999 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5644 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2421879189999991, 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046601900000000002, 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.522 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5644 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200521+0000", 2024-09-21T23:10:16.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200526+0000", 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-06 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200544+0000", 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7410000000000001e-05 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200559+0000", 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5435999999999999e-05 2024-09-21T23:10:16.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246920+0000", 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046360902000000002 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247055+0000", 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468200000000001 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247121+0000", 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6712999999999996e-05 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.525 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5645 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.2421291930000002, 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.04674565, 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5645 2024-09-21T23:10:16.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200579+0000", 2024-09-21T23:10:16.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2270000000000001e-06 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200585+0000", 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200605+0000", 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9675999999999999e-05 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200618+0000", 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509999999999999e-05 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247148+0000", 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046529451999999999 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247219+0000", 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0833000000000004e-05 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247324+0000", 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105122 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.530 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:16.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:16.531 INFO:teuthology.orchestra.run.smithi150.stdout:970 punch from 0x1dffb43 to 0x1e0cbe3, (0xd0a0 bytes) 2024-09-21T23:10:16.531 INFO:teuthology.orchestra.run.smithi150.stdout:971 write 0xbde401c thru 0xbded6cb (0x96b0 bytes) 2024-09-21T23:10:16.540 INFO:teuthology.orchestra.run.smithi150.stdout:972 trunc from 0xc339572 to 0xa761435 2024-09-21T23:10:16.541 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:16.542+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:16.567 INFO:teuthology.orchestra.run.smithi150.stdout:973 write 0x90c1f7f thru 0x90c35d3 (0x1655 bytes) 2024-09-21T23:10:16.595 INFO:teuthology.orchestra.run.smithi150.stdout:974 trunc from 0xa761435 to 0xf0cd32 2024-09-21T23:10:16.641 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:16.637+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:16.646 INFO:teuthology.orchestra.run.smithi150.stdout:975 write 0x67996a1 thru 0x679e218 (0x4b78 bytes) 2024-09-21T23:10:16.653 INFO:teuthology.orchestra.run.smithi150.stdout:976 read 0x3da389a thru 0x3db22b2 (0xea19 bytes) 2024-09-21T23:10:16.653 INFO:teuthology.orchestra.run.smithi150.stdout:977 punch from 0x1567a9d to 0x156f350, (0x78b3 bytes) 2024-09-21T23:10:16.667 INFO:teuthology.orchestra.run.smithi150.stdout:978 trunc from 0x679e219 to 0xba691fc 2024-09-21T23:10:16.669 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:16.670 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:16.670 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:16.670 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:16.679 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:16.679 INFO:teuthology.orchestra.run.smithi150.stdout:982 write 0x3033b8f thru 0x3038933 (0x4da5 bytes) 2024-09-21T23:10:16.684 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:16.692 INFO:teuthology.orchestra.run.smithi150.stdout:983 trunc from 0xba691fc to 0xe0db8e9 2024-09-21T23:10:16.705 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:16.705 INFO:teuthology.orchestra.run.smithi150.stdout:986 write 0x141cd24 thru 0x141ef3e (0x221b bytes) 2024-09-21T23:10:16.721 INFO:teuthology.orchestra.run.smithi150.stdout:987 write 0xecdd7a9 thru 0xece8f3f (0xb797 bytes) 2024-09-21T23:10:16.725 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:16.725 INFO:teuthology.orchestra.run.smithi150.stdout:989 punch from 0x555d6c2 to 0x5562a9d, (0x53db bytes) 2024-09-21T23:10:16.725 INFO:teuthology.orchestra.run.smithi150.stdout:990 read 0x1536bce thru 0x153db0c (0x6f3f bytes) 2024-09-21T23:10:16.726 INFO:teuthology.orchestra.run.smithi150.stdout:991 punch from 0xe16f8be to 0xe17db6a, (0xe2ac bytes) 2024-09-21T23:10:16.726 INFO:teuthology.orchestra.run.smithi150.stdout:992 writesame 0xbc4bd17 thru 0xbc55803 data_size 0x1(0x9aed bytes) 2024-09-21T23:10:16.743 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:16.737+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:16.784 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:16.784 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:16.800 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:16.800 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:16.800 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:16.800 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:16.801 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:16.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:16.815 INFO:teuthology.orchestra.run.smithi150.stdout:997 trunc from 0xece8f40 to 0x67ab94 2024-09-21T23:10:16.843 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:16.841+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:16.844 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:999 writesame 0xa2f3d3 thru 0xa3074e data_size 0x4(0x137c bytes) 2024-09-21T23:10:16.858 INFO:teuthology.orchestra.run.smithi150.stdout:1000 punch from 0x50de13 to 0x51b921, (0xdb0e bytes) 2024-09-21T23:10:16.858 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xe96b5ef 2024-09-21T23:10:16.892 INFO:teuthology.orchestra.run.smithi150.stdout:1001 trunc from 0xa3074f to 0xe96b5ef 2024-09-21T23:10:16.905 INFO:teuthology.orchestra.run.smithi150.stdout:1002 read 0xe5a6d7d thru 0xe5acae0 (0x5d64 bytes) 2024-09-21T23:10:16.905 INFO:teuthology.orchestra.run.smithi150.stdout:1003 write 0x3e873c5 thru 0x3e88c6f (0x18ab bytes) 2024-09-21T23:10:16.912 INFO:teuthology.orchestra.run.smithi150.stdout:1004 read 0x426540e thru 0x426c10f (0x6d02 bytes) 2024-09-21T23:10:16.913 INFO:teuthology.orchestra.run.smithi150.stdout:1005 write 0x238e11f thru 0x23928e5 (0x47c7 bytes) 2024-09-21T23:10:16.916 INFO:teuthology.orchestra.run.smithi150.stdout:1006 punch from 0x2abcace to 0x2ac5be9, (0x911b bytes) 2024-09-21T23:10:16.916 INFO:teuthology.orchestra.run.smithi150.stdout:1008 writesame 0xe55c67a thru 0xe56054f data_size 0x15(0x3ed6 bytes) 2024-09-21T23:10:16.922 INFO:teuthology.orchestra.run.smithi150.stdout:1009 read 0x4f67685 thru 0x4f6c415 (0x4d91 bytes) 2024-09-21T23:10:16.922 INFO:teuthology.orchestra.run.smithi150.stdout:1011 writesame 0x482560c thru 0x4833700 data_size 0x13(0xe0f5 bytes) 2024-09-21T23:10:16.944 INFO:teuthology.orchestra.run.smithi150.stdout:1012 write 0xea1670e thru 0xea1e128 (0x7a1b bytes) 2024-09-21T23:10:16.944 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:16.942+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:16.950 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:16.950 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3817 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.058259347, 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020519229, 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.951 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3817 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892626+0000", 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892631+0000", 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.57e-06 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892648+0000", 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7323e-05 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892663+0000", 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4610000000000001e-05 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908171+0000", 2024-09-21T23:10:16.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015508061 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908384+0000", 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000212945 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.913144+0000", 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047596000000000001 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3818 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 9895936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.058202507000001, 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016964216000000001, 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3818 2024-09-21T23:10:16.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892682+0000", 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892688+0000", 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7230000000000001e-06 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892707+0000", 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8423999999999998e-05 2024-09-21T23:10:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892720+0000", 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3900000000000001e-05 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908948+0000", 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016227075000000001 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909223+0000", 2024-09-21T23:10:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027500799999999998 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909645+0000", 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000422963 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3821 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 15663104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.058146906999999, 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905422000000001, 2024-09-21T23:10:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3821 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892738+0000", 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892744+0000", 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892763+0000", 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8916e-05 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892777+0000", 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4216000000000001e-05 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909722+0000", 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016944846999999999 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909918+0000", 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019664199999999999 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.910642+0000", 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072407799999999996 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.964 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6407 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1822322429999996, 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01587125, 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6407 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768653+0000", 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2729999999999999e-06 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768658+0000", 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768676+0000", 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8280000000000001e-05 2024-09-21T23:10:16.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768689+0000", 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2823000000000001e-05 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784031+0000", 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015341429 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784270+0000", 2024-09-21T23:10:16.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023918799999999999 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784523+0000", 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025289399999999998 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6409 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:16.969 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1821752110000006, 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016872036999999999, 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6409 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:16.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768710+0000", 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.877e-06 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768716+0000", 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768735+0000", 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8983000000000001e-05 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768750+0000", 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4246999999999999e-05 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785004+0000", 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016254497 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785251+0000", 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024662499999999999 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785581+0000", 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032997799999999998 2024-09-21T23:10:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6411 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1820617470000006, 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017242586000000001, 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6411 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768824+0000", 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6750000000000001e-06 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768847+0000", 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3706999999999999e-05 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768878+0000", 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000003e-05 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768899+0000", 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0468000000000001e-05 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785675+0000", 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016776669000000001 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785818+0000", 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014246199999999999 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786065+0000", 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024697500000000003 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6413 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1819509309999994, 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017559430000000001, 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6413 2024-09-21T23:10:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768934+0000", 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4929999999999999e-06 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768948+0000", 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768966+0000", 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8519999999999999e-05 2024-09-21T23:10:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768980+0000", 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3725e-05 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786147+0000", 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017167094000000001 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786280+0000", 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013344499999999999 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786492+0000", 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021196600000000001 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6415 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:16.983 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1818830560000002, 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905754999999999, 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6415 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769002+0000", 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769018+0000", 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6337999999999998e-05 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769036+0000", 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8042000000000001e-05 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769050+0000", 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786564+0000", 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017514538999999999 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786700+0000", 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786906+0000", 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000206689 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.987 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6418 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.181812549, 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018383022999999998, 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.988 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6418 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769072+0000", 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769078+0000", 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4219999999999999e-06 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769094+0000", 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5815e-05 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769107+0000", 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3305000000000001e-05 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787013+0000", 2024-09-21T23:10:16.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017906090999999999 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787208+0000", 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019526600000000001 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787454+0000", 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024600099999999998 2024-09-21T23:10:16.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6420 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1817610199999997, 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018780271000000001, 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6420 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:16.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769124+0000", 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3799999999999996e-07 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769129+0000", 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769146+0000", 2024-09-21T23:10:16.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198e-05 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769158+0000", 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1987999999999999e-05 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787568+0000", 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018410037000000001 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787697+0000", 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012926600000000001 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787903+0000", 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205692 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6421 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:16.997 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1817101300000008, 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019118928, 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6421 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:16.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:16.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769179+0000", 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.78e-06 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769194+0000", 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4576000000000001e-05 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769204+0000", 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0516999999999999e-05 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787991+0000", 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018786981000000001 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788130+0000", 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013833899999999999 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788293+0000", 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000162905 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6423 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1816634530000005, 2024-09-21T23:10:17.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019463588, 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6423 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769221+0000", 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999998e-07 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769226+0000", 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8799999999999999e-06 2024-09-21T23:10:17.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769235+0000", 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8300000000000002e-06 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769241+0000", 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2580000000000001e-06 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788361+0000", 2024-09-21T23:10:17.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019119873999999999 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788514+0000", 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152989 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788684+0000", 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016989899999999999 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6426 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1816340249999993, 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019969425999999998, 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6426 2024-09-21T23:10:17.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769253+0000", 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5490000000000001e-06 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769260+0000", 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5650000000000004e-06 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769270+0000", 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9180000000000006e-06 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788787+0000", 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019516977000000001 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788981+0000", 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019366300000000001 2024-09-21T23:10:17.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789219+0000", 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023821399999999999 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6427 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1816016559999998, 2024-09-21T23:10:17.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020632774999999999, 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6427 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769283+0000", 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1480000000000001e-06 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769288+0000", 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3819999999999997e-06 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769301+0000", 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2968e-05 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769309+0000", 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1040000000000007e-06 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789376+0000", 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020067406999999999 2024-09-21T23:10:17.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789621+0000", 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024461200000000001 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789915+0000", 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029415399999999998 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6429 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.181511982, 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021034428000000001, 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6429 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.018 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769374+0000", 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.906e-06 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769385+0000", 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1707e-05 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769410+0000", 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4102999999999999e-05 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769429+0000", 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9110000000000002e-05 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790009+0000", 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020580751000000001 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790186+0000", 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000176708 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790406+0000", 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022014300000000001 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6431 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1814238400000008, 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021448724999999998, 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.022 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6431 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:17.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769461+0000", 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.403e-06 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769468+0000", 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4069999999999998e-06 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769487+0000", 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769501+0000", 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4323e-05 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790548+0000", 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021047204 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790744+0000", 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019580199999999999 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790909+0000", 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164709 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.026 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6433 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1813638900000001, 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021886459, 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6433 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769521+0000", 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769527+0000", 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7100000000000004e-06 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769544+0000", 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7229999999999999e-05 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769558+0000", 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3567000000000001e-05 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790998+0000", 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021440516999999999 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791177+0000", 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017838899999999999 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791406+0000", 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022983300000000001 2024-09-21T23:10:17.030 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6436 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1813089970000004, 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022294958, 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6436 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769576+0000", 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769594+0000", 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8136999999999999e-05 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769612+0000", 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8063e-05 2024-09-21T23:10:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769626+0000", 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3706999999999999e-05 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791524+0000", 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021898407000000002 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791653+0000", 2024-09-21T23:10:17.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129116 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791870+0000", 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021640200000000001 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6437 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1812149139999999, 2024-09-21T23:10:17.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022700130999999998, 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6437 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769671+0000", 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6530000000000001e-06 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769680+0000", 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8830000000000001e-06 2024-09-21T23:10:17.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769701+0000", 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0883e-05 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769718+0000", 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6385999999999999e-05 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791966+0000", 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022248622999999999 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792154+0000", 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000187834 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792369+0000", 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021486899999999999 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.039 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6440 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1811456020000008, 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.023189891000000001, 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.040 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6440 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769740+0000", 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.327e-06 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769745+0000", 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6760000000000004e-06 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769762+0000", 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7187e-05 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769776+0000", 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3233e-05 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792416+0000", 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022640389 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792608+0000", 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019183899999999999 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792928+0000", 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032024000000000001 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.044 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:17.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:17.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 3.e --file /var/log/ceph/exp.3.e.4 2024-09-21T23:10:17.048 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:17.046+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1013 read 0x639628d thru 0x63a3194 (0xcf08 bytes) 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1015 writesame 0x693e958 thru 0x6949f1e data_size 0x29(0xb5c7 bytes) 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1016 writesame 0xbe480cf thru 0xbe50bf1 data_size 0x1f(0x8b23 bytes) 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1017 read 0x893c4b3 thru 0x8943102 (0x6c50 bytes) 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1019 read 0x4e65a47 thru 0x4e6ab97 (0x5151 bytes) 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1022 read 0x65677f3 thru 0x6571d07 (0xa515 bytes) 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1023 write 0x59b18a3 thru 0x59bcb2a (0xb288 bytes) 2024-09-21T23:10:17.048 INFO:teuthology.orchestra.run.smithi150.stdout:1024 trunc from 0xea1e129 to 0x8618410 2024-09-21T23:10:17.049 INFO:teuthology.orchestra.run.smithi150.stdout:1025 punch from 0x4ddb047 to 0x4de5b81, (0xab3a bytes) 2024-09-21T23:10:17.049 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:17.049 INFO:teuthology.orchestra.run.smithi150.stdout:1027 write 0x550184d thru 0x5501bc9 (0x37d bytes) 2024-09-21T23:10:17.049 INFO:teuthology.orchestra.run.smithi150.stdout:1029 punch from 0x3d22972 to 0x3d28bad, (0x623b bytes) 2024-09-21T23:10:17.049 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:17.049 INFO:teuthology.orchestra.run.smithi150.stdout:1031 write 0x729623e thru 0x729bbe4 (0x59a7 bytes) 2024-09-21T23:10:17.051 INFO:teuthology.orchestra.run.smithi150.stdout:1032 writesame 0xbbb8074 thru 0xbbc6890 data_size 0x1d(0xe81d bytes) 2024-09-21T23:10:17.051 INFO:teuthology.orchestra.run.smithi150.stdout:1033 read 0x8086cf6 thru 0x8090909 (0x9c14 bytes) 2024-09-21T23:10:17.052 INFO:teuthology.orchestra.run.smithi150.stdout:1034 write 0xa231b47 thru 0xa23b318 (0x97d2 bytes) 2024-09-21T23:10:17.060 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:17.060 INFO:teuthology.orchestra.run.smithi150.stdout:1036 read 0xae57951 thru 0xae615d3 (0x9c83 bytes) 2024-09-21T23:10:17.062 INFO:teuthology.orchestra.run.smithi150.stdout:1038 trunc from 0xbbc6891 to 0xb4f4c40 2024-09-21T23:10:17.095 INFO:teuthology.orchestra.run.smithi150.stdout:1039 writesame 0xc187e2d thru 0xc1977e4 data_size 0x83(0xf9b8 bytes) 2024-09-21T23:10:17.095 INFO:teuthology.orchestra.run.smithi150.stdout:1040 punch from 0x1f06fb to 0x1fc192, (0xba97 bytes) 2024-09-21T23:10:17.095 INFO:teuthology.orchestra.run.smithi150.stdout:1041 clone 11 order 18 su 131072 sc 3 2024-09-21T23:10:17.146 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:17.141+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:17.219 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:17.219 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:17.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:17.247 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:17.245+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:17.348 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:17.345+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:17.353 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:17.353 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:17.353 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:17.353 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:17.353 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:17.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:17.449 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:17.450+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:17.495 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.311552704, 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:17.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:10:17.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.499 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.311449617999999, 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.500 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:10:17.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:10:17.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.311382268999999, 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.507 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.236195333, 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.508 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:10:17.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:10:17.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.236139565, 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:10:17.512 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:17.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:10:17.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:10:17.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:10:17.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.236108026, 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.517 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:10:17.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:10:17.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.236075276999999, 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.521 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:10:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.236048384, 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.526 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:10:17.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.236023111, 2024-09-21T23:10:17.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:10:17.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:10:17.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.235991984, 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:10:17.538 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.235348628000001, 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.539 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230162700999999, 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.543 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:10:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.547 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230141218, 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230116932, 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.552 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:17.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:17.556 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230099265, 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:17.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:10:17.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:10:17.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230075680000001, 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230059665000001, 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.565 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:10:17.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:10:17.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230045834, 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.574 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.230014473000001, 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:17.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:10:17.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:10:17.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.229946051000001, 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.579 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:17.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:17.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:17.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:17.583 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:17.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:17.584 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:10:17.586 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:17.582+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:17.586 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:17.685 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:17.686+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:17.704 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone10 from 0xc1977e5 (overlap 0x67ab94) to 0x0 2024-09-21T23:10:17.735 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:17.735 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:17.735 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:17.735 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:17.753 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:17.754 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:17.754 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:17.787 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:17.787 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:17.787 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:17.785+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:17.811 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:17.812 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:17.814 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-21T23:10:17.823 INFO:teuthology.orchestra.run.smithi150.stdout:1042 writesame 0x3a1ff8 thru 0x3ac964 data_size 0x1(0xa96d bytes) 2024-09-21T23:10:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:17.832 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:17.832 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:17.874 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:17.874 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:17.878 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:17.878 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:17.883 INFO:teuthology.orchestra.run.smithi150.stdout:1043 read 0xa25e7c5 thru 0xa263073 (0x48af bytes) 2024-09-21T23:10:17.883 INFO:teuthology.orchestra.run.smithi150.stdout:1045 clone 12 order 21 su 16384 sc 11 2024-09-21T23:10:17.887 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:17.885+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:17.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:17.909 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:17.909 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:17.945 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:17.948 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:17.948 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:17.995 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:17.993+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:18.022 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:18.022 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:18.036 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:18.123 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:18.121+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:18.190 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:18.185+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:18.201 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:18.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:18.303 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:18.301+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:18.381 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:18.382 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:18.391 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:18.389+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:18.395 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:18.492 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:18.489+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:18.563 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:18.563 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:18.576 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:10:18.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:18.602 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:18.597+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:18.695 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:18.694+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone11 from 0xc1977e5 (overlap 0xc1977e5) to 0x3a13d91 2024-09-21T23:10:18.770 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:18.770 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:18.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:18.839 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:18.837+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:18.880 INFO:teuthology.orchestra.run.smithi175.stderr:Exporting 3.e info 3.e( v 31'36 (0'0,31'36] local-lis/les=21/22 n=4 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-21T23:10:18.888 INFO:teuthology.orchestra.run.smithi175.stderr:Read #3:727a85e1:::rbd_data.114912f5d0f.000000000000000e:7# 2024-09-21T23:10:18.891 INFO:teuthology.orchestra.run.smithi175.stderr:Read #3:727a85e1:::rbd_data.114912f5d0f.000000000000000e:head# 2024-09-21T23:10:18.892 INFO:teuthology.orchestra.run.smithi175.stderr:Read #3:7314555d:::rbd_data.1149b00fdffd.0000000000000003:head# 2024-09-21T23:10:18.896 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:18.893+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:18.913 INFO:teuthology.orchestra.run.smithi175.stderr:Read #3:775976dc:::rbd_header.114952d23f01:head# 2024-09-21T23:10:18.913 INFO:teuthology.orchestra.run.smithi175.stderr:Export successful 2024-09-21T23:10:18.957 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:18.957 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:18.980 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:19.023 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:19.021+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:19.097 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:19.093+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:19.165 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:19.165 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:19.179 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:19.223 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:19.221+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:19.253 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-21T23:10:19.256 INFO:teuthology.orchestra.run.smithi150.stdout:1048 read 0x46560e8 thru 0x46603ae (0xa2c7 bytes) 2024-09-21T23:10:19.256 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.256 INFO:teuthology.orchestra.run.smithi150.stdout:1050 read 0x4a74fb9 thru 0x4a7aa80 (0x5ac8 bytes) 2024-09-21T23:10:19.257 INFO:teuthology.orchestra.run.smithi150.stdout:1051 write 0xffdcb4 thru 0x1003611 (0x595e bytes) 2024-09-21T23:10:19.300 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:19.297+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:19.334 INFO:teuthology.orchestra.run.smithi150.stdout:1052 write 0x65bd794 thru 0x65c1b54 (0x43c1 bytes) 2024-09-21T23:10:19.351 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:19.351 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:19.367 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:19.403 INFO:teuthology.orchestra.run.smithi150.stdout:1054 trunc from 0xc1977e5 to 0xe2b703 2024-09-21T23:10:19.403 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:19.401+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:19.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 3.e 2024-09-21T23:10:19.501 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:19.497+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:19.538 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:19.538 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:19.553 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:19.582 INFO:teuthology.orchestra.run.smithi150.stdout:1055 writesame 0x7a1a54d thru 0x7a29ec9 data_size 0x11(0xf97d bytes) 2024-09-21T23:10:19.587 INFO:teuthology.orchestra.run.smithi150.stdout:1057 write 0x887bd3a thru 0x8886096 (0xa35d bytes) 2024-09-21T23:10:19.602 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:19.602+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:19.604 INFO:teuthology.orchestra.run.smithi150.stdout:1059 trunc from 0x8886097 to 0x116797c 2024-09-21T23:10:19.631 INFO:teuthology.orchestra.run.smithi150.stdout:1063 writesame 0x2e7508b thru 0x2e83471 data_size 0x1(0xe3e7 bytes) 2024-09-21T23:10:19.631 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.631 INFO:teuthology.orchestra.run.smithi150.stdout:1068 read 0x1adb83 thru 0x1b2ecd (0x534b bytes) 2024-09-21T23:10:19.631 INFO:teuthology.orchestra.run.smithi150.stdout:1069 read 0x4ae295 thru 0x4b3d29 (0x5a95 bytes) 2024-09-21T23:10:19.632 INFO:teuthology.orchestra.run.smithi150.stdout:1070 punch from 0x69211c to 0x69f845, (0xd729 bytes) 2024-09-21T23:10:19.648 INFO:teuthology.orchestra.run.smithi150.stdout:1071 writesame 0x7baa4ed thru 0x7bab88d data_size 0xf(0x13a1 bytes) 2024-09-21T23:10:19.649 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.649 INFO:teuthology.orchestra.run.smithi150.stdout:1073 write 0x7af9006 thru 0x7afeeeb (0x5ee6 bytes) 2024-09-21T23:10:19.670 INFO:teuthology.orchestra.run.smithi150.stdout:1074 write 0xbaf09b2 thru 0xbaf8f2f (0x857e bytes) 2024-09-21T23:10:19.680 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.680 INFO:teuthology.orchestra.run.smithi150.stdout:1077 read 0x79c6d5a thru 0x79cdde8 (0x708f bytes) 2024-09-21T23:10:19.681 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.689 INFO:teuthology.orchestra.run.smithi150.stdout:1081 trunc from 0xbaf8f30 to 0x53c4eee 2024-09-21T23:10:19.703 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:19.702+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:19.708 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:19.708 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:19.708 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:19.724 INFO:teuthology.orchestra.run.smithi150.stdout:1083 writesame 0x7ffee52 thru 0x80025f2 data_size 0x2f(0x37a1 bytes) 2024-09-21T23:10:19.738 INFO:teuthology.orchestra.run.smithi150.stdout:1084 write 0x9d36a2f thru 0x9d3aa7a (0x404c bytes) 2024-09-21T23:10:19.745 INFO:teuthology.orchestra.run.smithi150.stdout:1085 write 0x144bc6f thru 0x145a239 (0xe5cb bytes) 2024-09-21T23:10:19.748 INFO:teuthology.orchestra.run.smithi150.stdout:1087 punch from 0x1e04986 to 0x1e0cef1, (0x856b bytes) 2024-09-21T23:10:19.748 INFO:teuthology.orchestra.run.smithi150.stdout:1088 write 0x8e48683 thru 0x8e54758 (0xc0d6 bytes) 2024-09-21T23:10:19.756 INFO:teuthology.orchestra.run.smithi150.stdout:1089 writesame 0x3d1eb1b thru 0x3d29df3 data_size 0x1(0xb2d9 bytes) 2024-09-21T23:10:19.781 INFO:teuthology.orchestra.run.smithi150.stdout:1090 writesame 0x24d7c03 thru 0x24e7b96 data_size 0x16(0xff94 bytes) 2024-09-21T23:10:19.793 INFO:teuthology.orchestra.run.smithi150.stdout:1091 write 0x2da8613 thru 0x2da8dcb (0x7b9 bytes) 2024-09-21T23:10:19.795 INFO:teuthology.orchestra.run.smithi150.stdout:1092 writesame 0x90439d8 thru 0x904c23e data_size 0x1(0x8867 bytes) 2024-09-21T23:10:19.804 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:19.801+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:19.820 INFO:teuthology.orchestra.run.smithi150.stdout:1093 write 0x5e04a01 thru 0x5e0d0e0 (0x86e0 bytes) 2024-09-21T23:10:19.828 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.828 INFO:teuthology.orchestra.run.smithi150.stdout:1098 write 0x85a4b8f thru 0x85b1053 (0xc4c5 bytes) 2024-09-21T23:10:19.835 INFO:teuthology.orchestra.run.smithi150.stdout:1100 read 0x67594f3 thru 0x67653f4 (0xbf02 bytes) 2024-09-21T23:10:19.835 INFO:teuthology.orchestra.run.smithi150.stdout:1101 read 0x2948e1a thru 0x2949b87 (0xd6e bytes) 2024-09-21T23:10:19.835 INFO:teuthology.orchestra.run.smithi150.stdout:1102 read 0x8388f5b thru 0x838ac80 (0x1d26 bytes) 2024-09-21T23:10:19.835 INFO:teuthology.orchestra.run.smithi150.stdout:1103 punch from 0x659c585 to 0x65a3e9c, (0x7917 bytes) 2024-09-21T23:10:19.835 INFO:teuthology.orchestra.run.smithi150.stdout:1104 writesame 0x2022f96 thru 0x202aa5f data_size 0xd(0x7aca bytes) 2024-09-21T23:10:19.847 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:19.847 INFO:teuthology.orchestra.run.smithi150.stdout:1106 writesame 0x58a9683 thru 0x58b8ef5 data_size 0x6f(0xf873 bytes) 2024-09-21T23:10:19.865 INFO:teuthology.orchestra.run.smithi150.stdout:1107 write 0xa720aef thru 0xa72a16c (0x967e bytes) 2024-09-21T23:10:19.867 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:19.867 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:19.874 INFO:teuthology.orchestra.run.smithi150.stdout:1108 write 0x1b96fbf thru 0x1ba2f08 (0xbf4a bytes) 2024-09-21T23:10:19.880 INFO:teuthology.orchestra.run.smithi150.stdout:1110 read 0x2cbe3eb thru 0x2cc164a (0x3260 bytes) 2024-09-21T23:10:19.880 INFO:teuthology.orchestra.run.smithi150.stdout:1111 read 0x2c43ebc thru 0x2c4d8a2 (0x99e7 bytes) 2024-09-21T23:10:19.881 INFO:teuthology.orchestra.run.smithi150.stdout:1112 read 0x34fc906 thru 0x3506e7a (0xa575 bytes) 2024-09-21T23:10:19.882 INFO:teuthology.orchestra.run.smithi150.stdout:1113 trunc from 0xa72a16d to 0xa404a53 2024-09-21T23:10:19.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:19.907 INFO:teuthology.orchestra.run.smithi150.stdout:1114 trunc from 0xa404a53 to 0x4f31d73 2024-09-21T23:10:19.929 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:19.930+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:19.936 INFO:teuthology.orchestra.run.smithi150.stdout:1115 write 0x5a34f70 thru 0x5a44bdb (0xfc6c bytes) 2024-09-21T23:10:19.948 INFO:teuthology.orchestra.run.smithi150.stdout:1116 trunc from 0x5a44bdc to 0x6f3bb60 2024-09-21T23:10:19.949 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:19.956 INFO:teuthology.orchestra.run.smithi150.stdout:1118 punch from 0x45fe12a to 0x4607147, (0x901d bytes) 2024-09-21T23:10:19.975 INFO:teuthology.orchestra.run.smithi150.stdout:1121 trunc from 0x6f3bb60 to 0xe67afe7 2024-09-21T23:10:19.984 INFO:teuthology.orchestra.run.smithi150.stdout:1122 read 0x1404b6b thru 0x14060f2 (0x1588 bytes) 2024-09-21T23:10:19.985 INFO:teuthology.orchestra.run.smithi150.stdout:1124 writesame 0xc03f56d thru 0xc04e98a data_size 0x1(0xf41e bytes) 2024-09-21T23:10:20.005 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:20.001+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:20.012 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:20.013 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:20.020 INFO:teuthology.orchestra.run.smithi150.stdout:1127 write 0xea59887 thru 0xea6546b (0xbbe5 bytes) 2024-09-21T23:10:20.028 INFO:teuthology.orchestra.run.smithi150.stdout:1128 punch from 0x5d3bb65 to 0x5d499ea, (0xde85 bytes) 2024-09-21T23:10:20.028 INFO:teuthology.orchestra.run.smithi150.stdout:1130 clone 13 order 24 su 4194304 sc 15 2024-09-21T23:10:20.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:20.095 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:20.095 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:20.105 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:20.101+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:20.111 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:20.128 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:20.128 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:20.144 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:20.144 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:20.145 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:20.161 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:20.162 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:20.179 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:20.179 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:20.196 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:20.196 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5124 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.011960593, 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046086192999999998, 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5124 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186881+0000", 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.133e-06 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186887+0000", 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7939999999999999e-06 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186904+0000", 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7193e-05 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186918+0000", 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223027+0000", 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036108460000000002 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223201+0000", 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017483900000000001 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.232966+0000", 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097649039999999996 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5279 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.201 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010681953000001, 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045406334999999999, 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5279 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:20.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188160+0000", 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188167+0000", 2024-09-21T23:10:20.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0700000000000001e-06 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188188+0000", 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-05 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188205+0000", 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7444000000000001e-05 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233051+0000", 2024-09-21T23:10:20.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044846496999999999 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233167+0000", 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115307 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233565+0000", 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039829399999999999 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5280 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010612757000001, 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047857909999999997, 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5280 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188229+0000", 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.237e-06 2024-09-21T23:10:20.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188235+0000", 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188254+0000", 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9015999999999999e-05 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188268+0000", 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.395e-05 2024-09-21T23:10:20.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233608+0000", 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045339957 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233722+0000", 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011403700000000001 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236086+0000", 2024-09-21T23:10:20.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00236359 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5281 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010551256999999, 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048399091999999998, 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.210 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5281 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188291+0000", 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188298+0000", 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188315+0000", 2024-09-21T23:10:20.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7819999999999999e-05 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188329+0000", 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3383e-05 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236130+0000", 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047800827999999997 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236240+0000", 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110745 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236689+0000", 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044823600000000002 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5282 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010493378, 2024-09-21T23:10:20.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048602487, 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5282 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188348+0000", 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188355+0000", 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3029999999999996e-06 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188373+0000", 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8056000000000001e-05 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188386+0000", 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3536999999999999e-05 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236748+0000", 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048361617000000003 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236840+0000", 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2075000000000002e-05 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236950+0000", 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109755 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.218 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5283 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010350119, 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048844519000000003, 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5283 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.219 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188493+0000", 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3800000000000001e-06 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188503+0000", 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188526+0000", 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.304e-05 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188543+0000", 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7557000000000001e-05 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237020+0000", 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048476656 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237187+0000", 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016670199999999999 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237335+0000", 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014857100000000001 2024-09-21T23:10:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5284 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010268253, 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049117881000000002, 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.223 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5284 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188574+0000", 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188580+0000", 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188598+0000", 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8304e-05 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188611+0000", 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3329000000000001e-05 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237402+0000", 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048790838000000003 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237549+0000", 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146897 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237690+0000", 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141383 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5285 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010210967000001, 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049392706000000001, 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.227 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5285 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188631+0000", 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0400000000000005e-07 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188636+0000", 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8329999999999999e-06 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188655+0000", 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8246000000000001e-05 2024-09-21T23:10:20.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188670+0000", 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5440000000000001e-05 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237748+0000", 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049077975000000003 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237890+0000", 2024-09-21T23:10:20.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141953 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238022+0000", 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013235499999999999 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5286 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.010151863999999, 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049655168, 2024-09-21T23:10:20.231 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5286 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188690+0000", 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1769999999999999e-06 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188696+0000", 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5269999999999998e-06 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188713+0000", 2024-09-21T23:10:20.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7836e-05 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188726+0000", 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238077+0000", 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049350964999999997 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238228+0000", 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015027000000000001 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238344+0000", 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116403 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5287 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.235 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.009382276, 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049431115999999997, 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5287 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189460+0000", 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189470+0000", 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189491+0000", 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0743000000000001e-05 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189503+0000", 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2787e-05 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238395+0000", 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048891284 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238519+0000", 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124745 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238890+0000", 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037020400000000001 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.239 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5465 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.001586284, 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045516793999999999, 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5465 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197256+0000", 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197262+0000", 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197280+0000", 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197294+0000", 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3983e-05 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242417+0000", 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045122664999999999 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242550+0000", 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013329499999999999 2024-09-21T23:10:20.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242771+0000", 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022120399999999999 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5466 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.001528227, 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045852049999999998, 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.244 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5466 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:20.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197313+0000", 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197321+0000", 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197338+0000", 2024-09-21T23:10:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7824000000000001e-05 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197353+0000", 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.456e-05 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242853+0000", 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045500119999999998 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243010+0000", 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015682599999999999 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243165+0000", 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015463400000000001 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5467 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.001468984000001, 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046123232, 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5467 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197373+0000", 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197378+0000", 2024-09-21T23:10:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197397+0000", 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8793000000000001e-05 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197411+0000", 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3726999999999999e-05 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243249+0000", 2024-09-21T23:10:20.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045838320000000002 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243376+0000", 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012760199999999999 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243495+0000", 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011848100000000001 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5639 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.998657373, 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045343482999999997, 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5639 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.253 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200184+0000", 2024-09-21T23:10:20.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200190+0000", 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200209+0000", 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8563000000000002e-05 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200223+0000", 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4173e-05 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.244967+0000", 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044743567999999997 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245103+0000", 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013634300000000001 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245527+0000", 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042370199999999998 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5640 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.998561516000001, 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045524901, 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.257 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5640 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:20.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200280+0000", 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200287+0000", 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200307+0000", 2024-09-21T23:10:20.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0129999999999999e-05 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200321+0000", 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245576+0000", 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045255521 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245701+0000", 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012480100000000001 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245804+0000", 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102933 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5641 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:20.261 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.998498876999999, 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045750444000000001, 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5641 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:20.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200343+0000", 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200351+0000", 2024-09-21T23:10:20.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9160000000000001e-06 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200369+0000", 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8490000000000001e-05 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200383+0000", 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3234000000000001e-05 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245850+0000", 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045467883000000001 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245979+0000", 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128641 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246092+0000", 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113143 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5642 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.998438774, 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045975347999999999, 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5642 2024-09-21T23:10:20.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200403+0000", 2024-09-21T23:10:20.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200409+0000", 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6570000000000002e-06 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200427+0000", 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8306000000000001e-05 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200441+0000", 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4042999999999999e-05 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246144+0000", 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045702495000000003 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246277+0000", 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133831 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246377+0000", 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9835999999999998e-05 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5643 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.998381247999999, 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046409156, 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.270 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5643 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200460+0000", 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200466+0000", 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200483+0000", 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7776999999999999e-05 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200500+0000", 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6852999999999999e-05 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246582+0000", 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046081471999999998 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246725+0000", 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143184 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246869+0000", 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014361699999999999 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.274 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5644 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.998321174999999, 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046601900000000002, 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5644 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:20.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200521+0000", 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200526+0000", 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-06 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200544+0000", 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7410000000000001e-05 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200559+0000", 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5435999999999999e-05 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246920+0000", 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046360902000000002 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247055+0000", 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468200000000001 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247121+0000", 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6712999999999996e-05 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5645 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.998262449, 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.04674565, 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5645 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200579+0000", 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2270000000000001e-06 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200585+0000", 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200605+0000", 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9675999999999999e-05 2024-09-21T23:10:20.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200618+0000", 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509999999999999e-05 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247148+0000", 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046529451999999999 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247219+0000", 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0833000000000004e-05 2024-09-21T23:10:20.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247324+0000", 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105122 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.284 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:20.285 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:20.286 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:20.286 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:20.287 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:20.286+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:20.387 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:20.385+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:20.454 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:20.455 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:20.455 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:20.455 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:20.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:20.489 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:20.485+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:20.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-21T23:10:20.588 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:20.585+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:20.596 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:20.596 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:20.596 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:20.597 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:20.597 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:20.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:10:20.612 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:20.689 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:20.685+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:20.735 INFO:teuthology.orchestra.run.smithi150.stdout:leaving image image_client.0-clone12 intact 2024-09-21T23:10:20.738 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:20.738 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:20.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:20.738 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:20.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3817 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.846255676, 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020519229, 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3817 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892626+0000", 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:20.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892631+0000", 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.57e-06 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892648+0000", 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7323e-05 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892663+0000", 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4610000000000001e-05 2024-09-21T23:10:20.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908171+0000", 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015508061 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908384+0000", 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000212945 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.913144+0000", 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047596000000000001 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3818 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 9895936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.846198835999999, 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016964216000000001, 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3818 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892682+0000", 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892688+0000", 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7230000000000001e-06 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892707+0000", 2024-09-21T23:10:20.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8423999999999998e-05 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892720+0000", 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3900000000000001e-05 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908948+0000", 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016227075000000001 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909223+0000", 2024-09-21T23:10:20.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027500799999999998 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909645+0000", 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000422963 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3821 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 15663104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.846143236, 2024-09-21T23:10:20.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905422000000001, 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3821 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892738+0000", 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892744+0000", 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:10:20.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892763+0000", 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8916e-05 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892777+0000", 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4216000000000001e-05 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909722+0000", 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016944846999999999 2024-09-21T23:10:20.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909918+0000", 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019664199999999999 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.910642+0000", 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072407799999999996 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.751 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6407 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.970228572, 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01587125, 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6407 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768652+0000", 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768653+0000", 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2729999999999999e-06 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768658+0000", 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768676+0000", 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8280000000000001e-05 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768689+0000", 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2823000000000001e-05 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784031+0000", 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015341429 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784270+0000", 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023918799999999999 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.784523+0000", 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025289399999999998 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.755 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6409 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.970171540000001, 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016872036999999999, 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.756 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6409 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768710+0000", 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.877e-06 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768716+0000", 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768735+0000", 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8983000000000001e-05 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768750+0000", 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4246999999999999e-05 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785004+0000", 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016254497 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785251+0000", 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024662499999999999 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785581+0000", 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032997799999999998 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6411 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.970058076000001, 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017242586000000001, 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.760 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6411 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:20.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768824+0000", 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6750000000000001e-06 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768847+0000", 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3706999999999999e-05 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768878+0000", 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000003e-05 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768899+0000", 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0468000000000001e-05 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785675+0000", 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016776669000000001 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785818+0000", 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014246199999999999 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786065+0000", 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024697500000000003 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6413 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:20.764 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.96994726, 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017559430000000001, 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6413 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768934+0000", 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4929999999999999e-06 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768948+0000", 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:10:20.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768966+0000", 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8519999999999999e-05 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768980+0000", 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3725e-05 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786147+0000", 2024-09-21T23:10:20.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017167094000000001 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786280+0000", 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013344499999999999 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786492+0000", 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021196600000000001 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6415 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969879385, 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905754999999999, 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6415 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769002+0000", 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T23:10:20.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769018+0000", 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6337999999999998e-05 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769036+0000", 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8042000000000001e-05 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769050+0000", 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:20.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786564+0000", 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017514538999999999 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786700+0000", 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786906+0000", 2024-09-21T23:10:20.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000206689 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6418 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969808878, 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018383022999999998, 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6418 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769072+0000", 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769078+0000", 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4219999999999999e-06 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769094+0000", 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5815e-05 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769107+0000", 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3305000000000001e-05 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787013+0000", 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017906090999999999 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787208+0000", 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019526600000000001 2024-09-21T23:10:20.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787454+0000", 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024600099999999998 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6420 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969757349, 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018780271000000001, 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.777 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6420 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769124+0000", 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3799999999999996e-07 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769129+0000", 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769146+0000", 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198e-05 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769158+0000", 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1987999999999999e-05 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787568+0000", 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018410037000000001 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787697+0000", 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012926600000000001 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787903+0000", 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205692 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6421 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.781 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969706458999999, 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019118928, 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6421 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769179+0000", 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.78e-06 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769194+0000", 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4576000000000001e-05 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769204+0000", 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0516999999999999e-05 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787991+0000", 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018786981000000001 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788130+0000", 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013833899999999999 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788293+0000", 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000162905 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6423 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969659782000001, 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019463588, 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6423 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.786 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769221+0000", 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999998e-07 2024-09-21T23:10:20.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769226+0000", 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8799999999999999e-06 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769235+0000", 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8300000000000002e-06 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769241+0000", 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2580000000000001e-06 2024-09-21T23:10:20.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788361+0000", 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019119873999999999 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788514+0000", 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152989 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788684+0000", 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016989899999999999 2024-09-21T23:10:20.789 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6426 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969630354, 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019969425999999998, 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.790 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6426 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769253+0000", 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5490000000000001e-06 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769260+0000", 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5650000000000004e-06 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769270+0000", 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9180000000000006e-06 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788787+0000", 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019516977000000001 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788981+0000", 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019366300000000001 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789219+0000", 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023821399999999999 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6427 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969597985, 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020632774999999999, 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.794 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6427 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769283+0000", 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1480000000000001e-06 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769288+0000", 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3819999999999997e-06 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769301+0000", 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2968e-05 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769309+0000", 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1040000000000007e-06 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789376+0000", 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020067406999999999 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789621+0000", 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024461200000000001 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789915+0000", 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029415399999999998 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6429 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:20.798 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969508311, 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021034428000000001, 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6429 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:20.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769374+0000", 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.906e-06 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769385+0000", 2024-09-21T23:10:20.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1707e-05 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769410+0000", 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4102999999999999e-05 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769429+0000", 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9110000000000002e-05 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790009+0000", 2024-09-21T23:10:20.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020580751000000001 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790186+0000", 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000176708 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790406+0000", 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022014300000000001 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6431 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969420168999999, 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021448724999999998, 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6431 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.803 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769461+0000", 2024-09-21T23:10:20.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.403e-06 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769468+0000", 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4069999999999998e-06 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769487+0000", 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769501+0000", 2024-09-21T23:10:20.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4323e-05 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790548+0000", 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021047204 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790744+0000", 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019580199999999999 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790909+0000", 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164709 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6433 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969360219, 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021886459, 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.807 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6433 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:20.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769521+0000", 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769527+0000", 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7100000000000004e-06 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769544+0000", 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7229999999999999e-05 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769558+0000", 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3567000000000001e-05 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790998+0000", 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021440516999999999 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791177+0000", 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017838899999999999 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791406+0000", 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022983300000000001 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6436 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:20.811 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969305326000001, 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022294958, 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6436 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:20.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769576+0000", 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769594+0000", 2024-09-21T23:10:20.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8136999999999999e-05 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769612+0000", 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8063e-05 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769626+0000", 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3706999999999999e-05 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791524+0000", 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021898407000000002 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791653+0000", 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129116 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791870+0000", 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021640200000000001 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6437 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969211243, 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022700130999999998, 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6437 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769671+0000", 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6530000000000001e-06 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769680+0000", 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8830000000000001e-06 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769701+0000", 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0883e-05 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769718+0000", 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6385999999999999e-05 2024-09-21T23:10:20.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791966+0000", 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022248622999999999 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792154+0000", 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000187834 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792369+0000", 2024-09-21T23:10:20.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021486899999999999 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6440 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.969141930999999, 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.023189891000000001, 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:20.820 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6440 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769740+0000", 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.327e-06 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769745+0000", 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6760000000000004e-06 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769762+0000", 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7187e-05 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769776+0000", 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3233e-05 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792416+0000", 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022640389 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792608+0000", 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019183899999999999 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:20.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792928+0000", 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032024000000000001 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:20.824 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:20.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:20.826 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:20.826+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:20.926 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:20.921+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:20.993 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:20.993 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:20.993 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:20.993 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:21.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:21.026 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:21.021+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:21.082 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-21T23:10:21.127 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:21.126+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:21.141 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:21.141 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:21.141 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:21.141 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:21.141 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:21.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:21.245 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:21.246+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:21.301 INFO:teuthology.orchestra.run.smithi150.stdout:1131 read 0xd9f2877 thru 0xd9f657b (0x3d05 bytes) 2024-09-21T23:10:21.306 INFO:teuthology.orchestra.run.smithi150.stdout:1132 read 0xae467c2 thru 0xae51976 (0xb1b5 bytes) 2024-09-21T23:10:21.315 INFO:teuthology.orchestra.run.smithi150.stdout:1136 trunc from 0xea6546c to 0x86f44ca 2024-09-21T23:10:21.323 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.141143117, 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:10:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:10:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.141040030999999, 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:10:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:10:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.140972681999999, 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:10:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.065785746, 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:10:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.065729978, 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:10:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:10:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.065698439, 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:10:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:10:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:10:21.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.065665689999999, 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.350 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:10:21.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.065638797, 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:10:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.065613524, 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:10:21.358 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:10:21.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:10:21.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.065582397, 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:10:21.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:10:21.366 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.064939041000001, 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059753113999999, 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.371 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:10:21.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:10:21.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059731631, 2024-09-21T23:10:21.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:10:21.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:10:21.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:10:21.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:10:21.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:10:21.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:10:21.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:10:21.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059707345, 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.381 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:21.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:10:21.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059689678, 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:10:21.385 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.389 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059666093000001, 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059650078000001, 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059636247, 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:10:21.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:10:21.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:10:21.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059604886000001, 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.402 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:21.406 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.059536464000001, 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:21.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.410 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:21.411 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:21.411 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:21.411 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:21.378+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:21.411 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:21.412 INFO:teuthology.orchestra.run.smithi150.stdout:1137 read 0x7aeda6c thru 0x7aedb07 (0x9c bytes) 2024-09-21T23:10:21.412 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:21.413 INFO:teuthology.orchestra.run.smithi150.stdout:1139 read 0x31ef086 thru 0x31f1dcb (0x2d46 bytes) 2024-09-21T23:10:21.413 INFO:teuthology.orchestra.run.smithi150.stdout:1141 read 0x39d5c70 thru 0x39d6999 (0xd2a bytes) 2024-09-21T23:10:21.413 INFO:teuthology.orchestra.run.smithi150.stdout:1142 write 0x480f7d thru 0x481cb1 (0xd35 bytes) 2024-09-21T23:10:21.413 INFO:teuthology.orchestra.run.smithi150.stdout:1143 read 0x80749e0 thru 0x807d1c3 (0x87e4 bytes) 2024-09-21T23:10:21.422 INFO:teuthology.orchestra.run.smithi150.stdout:1144 write 0xca55459 thru 0xca5c9ef (0x7597 bytes) 2024-09-21T23:10:21.423 INFO:teuthology.orchestra.run.smithi150.stdout:1145 writesame 0x9371abf thru 0x93753fe data_size 0x20(0x3940 bytes) 2024-09-21T23:10:21.427 INFO:teuthology.orchestra.run.smithi150.stdout:1147 read 0xd26f0c thru 0xd29a33 (0x2b28 bytes) 2024-09-21T23:10:21.427 INFO:teuthology.orchestra.run.smithi150.stdout:1149 write 0x7aa5cd1 thru 0x7aa96f7 (0x3a27 bytes) 2024-09-21T23:10:21.429 INFO:teuthology.orchestra.run.smithi150.stdout:1150 write 0xc51d347 thru 0xc520a7a (0x3734 bytes) 2024-09-21T23:10:21.431 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:21.431 INFO:teuthology.orchestra.run.smithi150.stdout:1155 writesame 0x3545b2d thru 0x3552112 data_size 0x2(0xc5e6 bytes) 2024-09-21T23:10:21.463 INFO:teuthology.orchestra.run.smithi150.stdout:1157 write 0x85856e1 thru 0x8591720 (0xc040 bytes) 2024-09-21T23:10:21.467 INFO:teuthology.orchestra.run.smithi150.stdout:1158 read 0x2f9e744 thru 0x2fa6929 (0x81e6 bytes) 2024-09-21T23:10:21.468 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:21.480 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:21.477+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:21.480 INFO:teuthology.orchestra.run.smithi150.stdout:1160 trunc from 0xca5c9f0 to 0x4be1e47 2024-09-21T23:10:21.497 INFO:teuthology.orchestra.run.smithi150.stdout:1162 read 0xb795e8 thru 0xb7d682 (0x409b bytes) 2024-09-21T23:10:21.498 INFO:teuthology.orchestra.run.smithi150.stdout:1163 read 0x387a0bd thru 0x387a7d1 (0x715 bytes) 2024-09-21T23:10:21.499 INFO:teuthology.orchestra.run.smithi150.stdout:1164 read 0xb4f634 thru 0xb5a5c9 (0xaf96 bytes) 2024-09-21T23:10:21.500 INFO:teuthology.orchestra.run.smithi150.stdout:1165 punch from 0x2a067fc to 0x2a09af6, (0x32fa bytes) 2024-09-21T23:10:21.500 INFO:teuthology.orchestra.run.smithi150.stdout:1167 punch from 0x15df226 to 0x15e86a8, (0x9482 bytes) 2024-09-21T23:10:21.507 INFO:teuthology.orchestra.run.smithi150.stdout:1169 write 0x4db30ba thru 0x4db9047 (0x5f8e bytes) 2024-09-21T23:10:21.510 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:21.542 INFO:teuthology.orchestra.run.smithi150.stdout:1171 write 0xe1c2991 thru 0xe1ce2c0 (0xb930 bytes) 2024-09-21T23:10:21.545 INFO:teuthology.orchestra.run.smithi150.stdout:1172 write 0xc7feaef thru 0xc8016a5 (0x2bb7 bytes) 2024-09-21T23:10:21.547 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:21.547 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:21.547 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:21.548 INFO:teuthology.orchestra.run.smithi150.stdout:1173 write 0xc117444 thru 0xc121e3b (0xa9f8 bytes) 2024-09-21T23:10:21.558 INFO:teuthology.orchestra.run.smithi150.stdout:1174 trunc from 0xe1ce2c1 to 0xa64c8a6 2024-09-21T23:10:21.566 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:21.571 INFO:teuthology.orchestra.run.smithi150.stdout:1175 punch from 0x49a62f7 to 0x49a9ba8, (0x38b1 bytes) 2024-09-21T23:10:21.571 INFO:teuthology.orchestra.run.smithi150.stdout:1177 writesame 0xd6004a thru 0xd6d89b data_size 0x1(0xd852 bytes) 2024-09-21T23:10:21.591 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:10:21.591 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:21.589+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:21.599 INFO:teuthology.orchestra.run.smithi175.stdout: marking collection for removal 2024-09-21T23:10:21.599 INFO:teuthology.orchestra.run.smithi175.stdout:setting '_remove' omap key 2024-09-21T23:10:21.600 INFO:teuthology.orchestra.run.smithi175.stdout:finish_remove_pgs 3.e_head removing 3.e 2024-09-21T23:10:21.602 INFO:teuthology.orchestra.run.smithi175.stdout:Remove successful 2024-09-21T23:10:21.619 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:21.619 INFO:teuthology.orchestra.run.smithi150.stdout:1179 read 0x7281f1e thru 0x7282248 (0x32b bytes) 2024-09-21T23:10:21.621 INFO:teuthology.orchestra.run.smithi150.stdout:1180 read 0x846ce57 thru 0x8476f7d (0xa127 bytes) 2024-09-21T23:10:21.622 INFO:teuthology.orchestra.run.smithi150.stdout:1182 read 0x12d0f14 thru 0x12d5210 (0x42fd bytes) 2024-09-21T23:10:21.626 INFO:teuthology.orchestra.run.smithi150.stdout:1183 trunc from 0xa64c8a6 to 0x79381bd 2024-09-21T23:10:21.646 INFO:teuthology.orchestra.run.smithi150.stdout:1186 write 0x9841cd8 thru 0x9846f4f (0x5278 bytes) 2024-09-21T23:10:21.648 INFO:teuthology.orchestra.run.smithi150.stdout:1187 punch from 0x3de4e02 to 0x3de641a, (0x1618 bytes) 2024-09-21T23:10:21.648 INFO:teuthology.orchestra.run.smithi150.stdout:1188 read 0x486ff82 thru 0x4871260 (0x12df bytes) 2024-09-21T23:10:21.648 INFO:teuthology.orchestra.run.smithi150.stdout:1190 clone 14 order 22 su 262144 sc 3 2024-09-21T23:10:21.681 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:21.677+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:21.699 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:21.699 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:21.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:21.782 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:21.782+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:21.880 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:21.880 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:21.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:21.880 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:21.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.880 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12737 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2043904~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.880 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7809047959999997, 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035132690000000001, 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12737 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:21.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097394+0000", 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097397+0000", 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9890000000000001e-06 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097400+0000", 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132379+0000", 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034979237000000003 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132438+0000", 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8183999999999997e-05 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132526+0000", 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8171999999999995e-05 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12738 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2058240~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7808945089999995, 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035265405999999999, 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12738 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.885 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097404+0000", 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-21T23:10:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097405+0000", 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097408+0000", 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.971e-06 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097410+0000", 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132558+0000", 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035147993000000002 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132611+0000", 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2293000000000002e-05 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132669+0000", 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8139e-05 2024-09-21T23:10:21.888 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12739 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2072576~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7808844429999997, 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035408621000000001, 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.889 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12739 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097414+0000", 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097415+0000", 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.046e-06 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097418+0000", 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9629999999999998e-06 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097424+0000", 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.891e-06 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132706+0000", 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035282154000000003 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132762+0000", 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6475999999999999e-05 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132822+0000", 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9877999999999998e-05 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12740 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2086912~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7808683789999993, 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035536514999999998, 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12740 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6399999999999998e-07 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097432+0000", 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.889e-06 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097438+0000", 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9270000000000002e-06 2024-09-21T23:10:21.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097442+0000", 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4340000000000002e-06 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132856+0000", 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035413922 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132906+0000", 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9997999999999998e-05 2024-09-21T23:10:21.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132966+0000", 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9981000000000001e-05 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12797 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1313792~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802847879999995, 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034983512000000001, 2024-09-21T23:10:21.897 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12797 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098015+0000", 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1850000000000001e-06 2024-09-21T23:10:21.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098018+0000", 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1540000000000002e-06 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098020+0000", 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6390000000000001e-06 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132848+0000", 2024-09-21T23:10:21.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034828143999999998 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132909+0000", 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0825999999999997e-05 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132997+0000", 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7348999999999993e-05 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12798 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1328128~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802741900000001, 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035158699000000002, 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12798 2024-09-21T23:10:21.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098025+0000", 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0440000000000001e-06 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098028+0000", 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098031+0000", 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7499999999999999e-06 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133037+0000", 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035005938 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133095+0000", 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8208000000000002e-05 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133182+0000", 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7559000000000007e-05 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12799 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1342464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802635549999994, 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035342140000000001, 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12799 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:21.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098035+0000", 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098036+0000", 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.172e-06 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098040+0000", 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8319999999999999e-06 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098042+0000", 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8509999999999999e-06 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133220+0000", 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035177096999999997 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133288+0000", 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8534000000000005e-05 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133376+0000", 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8441000000000002e-05 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12800 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1356800~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:21.910 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802524389999999, 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03547894, 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12800 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098046+0000", 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098047+0000", 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:10:21.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098051+0000", 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.687e-06 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098053+0000", 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.689e-06 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133401+0000", 2024-09-21T23:10:21.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035347575999999999 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133446+0000", 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4962000000000001e-05 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133524+0000", 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8568999999999999e-05 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12801 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1371136~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802411790000008, 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035624792000000002, 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12801 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.915 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:21.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098058+0000", 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.105e-06 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098062+0000", 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.844e-06 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098065+0000", 2024-09-21T23:10:21.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7310000000000002e-06 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133557+0000", 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035492255 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133605+0000", 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8569000000000002e-05 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133682+0000", 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6070999999999997e-05 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12802 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1385472~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802300510000002, 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035748412, 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.919 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.920 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12802 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098069+0000", 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:10:21.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098072+0000", 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.033e-06 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098075+0000", 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8490000000000002e-06 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133709+0000", 2024-09-21T23:10:21.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035633962999999998 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133753+0000", 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3809e-05 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133816+0000", 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3301999999999998e-05 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12803 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1399808~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802189530000003, 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035862430000000001, 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12803 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098080+0000", 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098083+0000", 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0479999999999999e-06 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098087+0000", 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2030000000000001e-06 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133842+0000", 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035755012000000003 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133885+0000", 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000002e-05 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133941+0000", 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6648999999999999e-05 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.928 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12804 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1414144~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7802082339999998, 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035982214999999998, 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12804 2024-09-21T23:10:21.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098091+0000", 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2449999999999999e-06 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098094+0000", 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0120000000000001e-06 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098097+0000", 2024-09-21T23:10:21.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7199999999999998e-06 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133968+0000", 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035871003999999998 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134016+0000", 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8368999999999997e-05 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134072+0000", 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5649999999999997e-05 2024-09-21T23:10:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12805 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1428480~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7801980359999998, 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036102355000000003, 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12805 2024-09-21T23:10:21.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098101+0000", 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098110+0000", 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1290000000000002e-06 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098113+0000", 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8310000000000001e-06 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134100+0000", 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035986481000000001 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134144+0000", 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3890000000000002e-05 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134202+0000", 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8613000000000003e-05 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12806 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1442816~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7801817129999993, 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036273185999999999, 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12806 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098117+0000", 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098118+0000", 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.124e-06 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098121+0000", 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:10:21.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098124+0000", 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8320000000000002e-06 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134229+0000", 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036105406 2024-09-21T23:10:21.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134295+0000", 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6282e-05 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134389+0000", 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4168999999999999e-05 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.941 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12807 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1457152~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7801694230000003, 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037538491, 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12807 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098129+0000", 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2100000000000001e-07 2024-09-21T23:10:21.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098130+0000", 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1149999999999999e-06 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098133+0000", 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.258e-06 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098136+0000", 2024-09-21T23:10:21.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7810000000000001e-06 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135467+0000", 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037331371000000002 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135549+0000", 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2211000000000003e-05 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135667+0000", 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117534 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12808 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1471488~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7801588959999997, 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037746517, 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12808 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:21.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098142+0000", 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098145+0000", 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098148+0000", 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7750000000000001e-06 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135714+0000", 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037566612999999999 2024-09-21T23:10:21.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135788+0000", 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4000999999999998e-05 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135886+0000", 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7272e-05 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12809 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1485824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7801471430000007, 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037999241000000003, 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12809 2024-09-21T23:10:21.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1e-07 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098152+0000", 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0130000000000001e-06 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098155+0000", 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0290000000000001e-06 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098158+0000", 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.565e-06 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135936+0000", 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037778467000000003 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136047+0000", 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011086099999999999 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136150+0000", 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103096 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12810 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1500160~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7801373819999995, 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038237449, 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12810 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098162+0000", 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.189e-06 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098165+0000", 2024-09-21T23:10:21.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9780000000000001e-06 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098168+0000", 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136187+0000", 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038019325999999999 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136272+0000", 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4497000000000001e-05 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136398+0000", 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012611600000000001 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12811 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1514496~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:21.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.7801269150000003, 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038486382, 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12811 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098172+0000", 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.24e-06 2024-09-21T23:10:21.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098175+0000", 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.976e-06 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098178+0000", 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8200000000000001e-06 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136448+0000", 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038269965000000003 2024-09-21T23:10:21.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136543+0000", 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5069999999999996e-05 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136657+0000", 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114071 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21978 3.1f 3:fa161337:::rbd_data.1149c115bf92.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:21.963 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1449946879999997, 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052821716999999997, 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21978 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:21.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733307+0000", 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733321+0000", 2024-09-21T23:10:21.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4049999999999999e-05 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733357+0000", 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6363999999999999e-05 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733375+0000", 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733493+0000", 2024-09-21T23:10:21.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117642 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733629+0000", 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000136596 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.742696+0000", 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090670260000000006 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:21.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.785924+0000", 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043228124 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.785957+0000", 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2252999999999998e-05 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.786048+0000", 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1731999999999995e-05 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:21.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.786125+0000", 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6606999999999993e-05 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:21.969 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:21.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:21.971 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:21.973+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:22.072 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:22.069+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:22.098 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:22.098 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:22.098 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:22.098 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:22.112 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:22.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.e.4 2024-09-21T23:10:22.199 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:22.197+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:22.273 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:22.269+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:22.280 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:22.280 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:22.280 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:22.280 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:22.280 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:22.287 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:22.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:22.374 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:22.373+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:22.435 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:22.435 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:22.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:22.452 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:22.452 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:22.452 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:22.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:22.469 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:22.469 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:22.474 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:22.469+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:22.486 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:22.486 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:22.486 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:22.507 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:22.507 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:22.525 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:22.525 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:22.546 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:22.546 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:22.549 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:22.550 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:22.587 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:22.587 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:22.588 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:22.585+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:22.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:22.691 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:22.689+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:22.775 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:22.775 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:22.777 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:22.773+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:22.791 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone13 from 0x9846f50 (overlap 0x4be1e47) to 0x0 2024-09-21T23:10:22.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:22.879 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:22.877+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:22.979 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:22.977+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:23.001 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:23.001 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:23.008 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-21T23:10:23.009 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:23.099 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:23.097+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:23.181 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:23.177+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:23.187 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:23.187 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:23.203 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:23.291 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:23.289+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:23.366 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:23.367 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:23.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:23.384 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:23.381+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:23.481 INFO:teuthology.orchestra.run.smithi150.stdout:1191 read 0x1ae62e7 thru 0x1af06dc (0xa3f6 bytes) 2024-09-21T23:10:23.484 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:23.481+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:23.506 INFO:teuthology.orchestra.run.smithi150.stdout:1192 writesame 0xd72275c thru 0xd72811c data_size 0x9(0x59c1 bytes) 2024-09-21T23:10:23.506 INFO:teuthology.orchestra.run.smithi150.stdout:1194 write 0x55892a8 thru 0x55903c2 (0x711b bytes) 2024-09-21T23:10:23.514 INFO:teuthology.orchestra.run.smithi150.stdout:1195 write 0x21c1a81 thru 0x21c9239 (0x77b9 bytes) 2024-09-21T23:10:23.523 INFO:teuthology.orchestra.run.smithi150.stdout:1196 punch from 0x4b6c01b to 0x4b72e8b, (0x6e70 bytes) 2024-09-21T23:10:23.523 INFO:teuthology.orchestra.run.smithi150.stdout:1200 read 0x4a2207d thru 0x4a284e8 (0x646c bytes) 2024-09-21T23:10:23.524 INFO:teuthology.orchestra.run.smithi150.stdout:1202 write 0x662683 thru 0x66f87f (0xd1fd bytes) 2024-09-21T23:10:23.555 INFO:teuthology.orchestra.run.smithi150.stdout:1203 write 0x303c35a thru 0x3040a87 (0x472e bytes) 2024-09-21T23:10:23.556 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:23.556 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:23.568 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.568 INFO:teuthology.orchestra.run.smithi150.stdout:1207 punch from 0x5637294 to 0x563ca52, (0x57be bytes) 2024-09-21T23:10:23.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:23.575 INFO:teuthology.orchestra.run.smithi150.stdout:1208 trunc from 0xd72811d to 0x8c30fa0 2024-09-21T23:10:23.584 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:23.581+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:23.596 INFO:teuthology.orchestra.run.smithi150.stdout:1209 writesame 0x423c5a7 thru 0x4248442 data_size 0x16(0xbe9c bytes) 2024-09-21T23:10:23.608 INFO:teuthology.orchestra.run.smithi150.stdout:1212 punch from 0x82e5be0 to 0x82ec7f1, (0x6c11 bytes) 2024-09-21T23:10:23.608 INFO:teuthology.orchestra.run.smithi150.stdout:1213 read 0x4905051 thru 0x49105c1 (0xb571 bytes) 2024-09-21T23:10:23.609 INFO:teuthology.orchestra.run.smithi150.stdout:1216 writesame 0x3155841 thru 0x316156c data_size 0x2(0xbd2c bytes) 2024-09-21T23:10:23.613 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:10:23.633 INFO:teuthology.orchestra.run.smithi150.stdout:1218 read 0x6ea9579 thru 0x6eb620f (0xcc97 bytes) 2024-09-21T23:10:23.634 INFO:teuthology.orchestra.run.smithi150.stdout:1219 write 0x653e8c8 thru 0x6542dfa (0x4533 bytes) 2024-09-21T23:10:23.641 INFO:teuthology.orchestra.run.smithi150.stdout:1220 read 0x1dced16 thru 0x1ddc7be (0xdaa9 bytes) 2024-09-21T23:10:23.642 INFO:teuthology.orchestra.run.smithi150.stdout:1221 read 0x3a0de3e thru 0x3a11b5f (0x3d22 bytes) 2024-09-21T23:10:23.642 INFO:teuthology.orchestra.run.smithi150.stdout:1223 writesame 0x89b7955 thru 0x89bfd50 data_size 0x2(0x83fc bytes) 2024-09-21T23:10:23.652 INFO:teuthology.orchestra.run.smithi150.stdout:1224 read 0x2c322df thru 0x2c38384 (0x60a6 bytes) 2024-09-21T23:10:23.653 INFO:teuthology.orchestra.run.smithi150.stdout:1226 write 0x8a27e35 thru 0x8a34435 (0xc601 bytes) 2024-09-21T23:10:23.655 INFO:teuthology.orchestra.run.smithi150.stdout:1227 write 0x5b8fd86 thru 0x5b91430 (0x16ab bytes) 2024-09-21T23:10:23.656 INFO:teuthology.orchestra.run.smithi150.stdout:1228 read 0x55ef046 thru 0x55f1ccd (0x2c88 bytes) 2024-09-21T23:10:23.668 INFO:teuthology.orchestra.run.smithi150.stdout:1229 trunc from 0x8c30fa0 to 0x21ec101 2024-09-21T23:10:23.685 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:23.685+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:23.695 INFO:teuthology.orchestra.run.smithi150.stdout:1230 read 0xf9b39e thru 0xfa8a54 (0xd6b7 bytes) 2024-09-21T23:10:23.696 INFO:teuthology.orchestra.run.smithi150.stdout:1231 writesame 0x2083e03 thru 0x208fc24 data_size 0x1(0xbe22 bytes) 2024-09-21T23:10:23.729 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:23.729 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:23.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:23.748 INFO:teuthology.orchestra.run.smithi150.stdout:1232 write 0xc4157f7 thru 0xc421dce (0xc5d8 bytes) 2024-09-21T23:10:23.752 INFO:teuthology.orchestra.run.smithi150.stdout:1233 read 0x1a26d08 thru 0x1a28e9e (0x2197 bytes) 2024-09-21T23:10:23.753 INFO:teuthology.orchestra.run.smithi150.stdout:1237 punch from 0x2025a51 to 0x2035901, (0xfeb0 bytes) 2024-09-21T23:10:23.753 INFO:teuthology.orchestra.run.smithi150.stdout:1241 write 0x4947352 thru 0x494f2b1 (0x7f60 bytes) 2024-09-21T23:10:23.757 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.758 INFO:teuthology.orchestra.run.smithi150.stdout:1243 write 0xb5f4f24 thru 0xb5f8fda (0x40b7 bytes) 2024-09-21T23:10:23.760 INFO:teuthology.orchestra.run.smithi150.stdout:1245 write 0x3b8cb5f thru 0x3b9baea (0xef8c bytes) 2024-09-21T23:10:23.764 INFO:teuthology.orchestra.run.smithi150.stdout:1246 write 0x92abfb2 thru 0x92b4506 (0x8555 bytes) 2024-09-21T23:10:23.768 INFO:teuthology.orchestra.run.smithi150.stdout:1248 read 0xa8966a6 thru 0xa8a631a (0xfc75 bytes) 2024-09-21T23:10:23.768 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.768 INFO:teuthology.orchestra.run.smithi150.stdout:1250 write 0x772ca78 thru 0x7736e7a (0xa403 bytes) 2024-09-21T23:10:23.785 INFO:teuthology.orchestra.run.smithi150.stdout:1251 writesame 0xe414c73 thru 0xe422482 data_size 0x8(0xd810 bytes) 2024-09-21T23:10:23.785 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.785 INFO:teuthology.orchestra.run.smithi150.stdout:1253 punch from 0xe0d83d1 to 0xe0df1e0, (0x6e0f bytes) 2024-09-21T23:10:23.785 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.785 INFO:teuthology.orchestra.run.smithi150.stdout:1255 write 0x387c293 thru 0x3885902 (0x9670 bytes) 2024-09-21T23:10:23.789 INFO:teuthology.orchestra.run.smithi150.stdout:1256 read 0xa8b2443 thru 0xa8b8947 (0x6505 bytes) 2024-09-21T23:10:23.789 INFO:teuthology.orchestra.run.smithi150.stdout:1257 writesame 0xc362c4d thru 0xc36fc54 data_size 0x1c(0xd008 bytes) 2024-09-21T23:10:23.790 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:23.789+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:23.791 INFO:teuthology.orchestra.run.smithi150.stdout:1258 punch from 0x223b975 to 0x223fc8a, (0x4315 bytes) 2024-09-21T23:10:23.791 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.792 INFO:teuthology.orchestra.run.smithi150.stdout:1260 writesame 0x18b8f60 thru 0x18c461b data_size 0xa(0xb6bc bytes) 2024-09-21T23:10:23.794 INFO:teuthology.orchestra.run.smithi150.stdout:1261 read 0x1539043 thru 0x153bf22 (0x2ee0 bytes) 2024-09-21T23:10:23.795 INFO:teuthology.orchestra.run.smithi150.stdout:1264 punch from 0x3cb9b29 to 0x3cc665d, (0xcb34 bytes) 2024-09-21T23:10:23.795 INFO:teuthology.orchestra.run.smithi150.stdout:1266 punch from 0x37667d0 to 0x3768e5c, (0x268c bytes) 2024-09-21T23:10:23.803 INFO:teuthology.orchestra.run.smithi150.stdout:1268 writesame 0xe67bba4 thru 0xe680900 data_size 0x11(0x4d5d bytes) 2024-09-21T23:10:23.803 INFO:teuthology.orchestra.run.smithi150.stdout:1269 write 0xa267e45 thru 0xa275f71 (0xe12d bytes) 2024-09-21T23:10:23.808 INFO:teuthology.orchestra.run.smithi150.stdout:1270 writesame 0x8c29dc6 thru 0x8c33920 data_size 0x1b(0x9b5b bytes) 2024-09-21T23:10:23.829 INFO:teuthology.orchestra.run.smithi150.stdout:1271 trunc from 0xe680901 to 0x26adc14 2024-09-21T23:10:23.844 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.844 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:23.844 INFO:teuthology.orchestra.run.smithi150.stdout:1276 clone 15 order 18 su 32768 sc 10 2024-09-21T23:10:23.887 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:23.885+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:23.910 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:23.911 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:23.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:23.988 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:23.985+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:24.088 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:24.088 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:24.089 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:24.089+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:24.102 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:24.190 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:24.189+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:24.224 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:24.224 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:24.240 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:24.248 INFO:teuthology.orchestra.run.smithi175.stdout:get_pg_num_history pg_num_history pg_num_history(e32 pg_nums {1={14=1},2={17=8},3={21=32}} deleted_pools ) 2024-09-21T23:10:24.248 INFO:teuthology.orchestra.run.smithi175.stdout:Importing pgid 3.e 2024-09-21T23:10:24.248 INFO:teuthology.orchestra.run.smithi175.stdout:Write #3:727a85e1:::rbd_data.114912f5d0f.000000000000000e:7# 2024-09-21T23:10:24.251 INFO:teuthology.orchestra.run.smithi175.stdout:Write #3:727a85e1:::rbd_data.114912f5d0f.000000000000000e:head# 2024-09-21T23:10:24.251 INFO:teuthology.orchestra.run.smithi175.stdout:snapset 7=[]:{7=[7]} 2024-09-21T23:10:24.252 INFO:teuthology.orchestra.run.smithi175.stdout:Write #3:7314555d:::rbd_data.1149b00fdffd.0000000000000003:head# 2024-09-21T23:10:24.262 INFO:teuthology.orchestra.run.smithi175.stdout:snapset 0=[]:{} 2024-09-21T23:10:24.290 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:24.289+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:24.301 INFO:teuthology.orchestra.run.smithi175.stdout:Write #3:775976dc:::rbd_header.114952d23f01:head# 2024-09-21T23:10:24.301 INFO:teuthology.orchestra.run.smithi175.stdout:snapset 0=[]:{} 2024-09-21T23:10:24.302 INFO:teuthology.orchestra.run.smithi175.stdout:write_pg epoch 31 info 3.e( v 31'36 (0'0,31'36] local-lis/les=21/22 n=4 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-21T23:10:24.302 INFO:teuthology.orchestra.run.smithi175.stdout:Import successful 2024-09-21T23:10:24.355 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:24.355 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:24.369 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:24.391 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:24.385+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:24.491 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:24.489+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:24.539 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:24.539 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:24.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:24.553 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:24.597 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:24.597+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:24.611 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:10:24.678 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:24.678 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:24.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:24.694 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:24.689+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:24.710 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:24.710 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:24.728 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:24.729 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:24.746 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:24.746 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:24.753 INFO:teuthology.orchestra.run.smithi150.stdout:leaving image image_client.0-clone14 intact 2024-09-21T23:10:24.763 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:24.763 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:24.763 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:24.782 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:24.782 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:24.794 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:24.789+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:24.800 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:24.800 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:24.819 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:24.819 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:24.822 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:24.822 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:24.866 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-21T23:10:24.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /var/log/ceph/exp.3.e.4 2024-09-21T23:10:24.868 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:24.869 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:24.869 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:24.869 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:24.869 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:24.870 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:24.870 INFO:teuthology.orchestra.run.smithi150.stdout:1279 read 0x18c7d09 thru 0x18d6add (0xedd5 bytes) 2024-09-21T23:10:24.870 INFO:teuthology.orchestra.run.smithi150.stdout:1280 punch from 0x22b7ff7 to 0x22b874f, (0x758 bytes) 2024-09-21T23:10:24.882 INFO:teuthology.orchestra.run.smithi150.stdout:1281 writesame 0x3cfabee thru 0x3d06b17 data_size 0x1(0xbf2a bytes) 2024-09-21T23:10:24.882 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:24.882 INFO:teuthology.orchestra.run.smithi150.stdout:1284 read 0x2ab046f thru 0x2ab3108 (0x2c9a bytes) 2024-09-21T23:10:24.882 INFO:teuthology.orchestra.run.smithi150.stdout:1285 writesame 0x2830952 thru 0x283ef17 data_size 0x1(0xe5c6 bytes) 2024-09-21T23:10:24.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:24.930 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:24.929+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:24.944 INFO:teuthology.orchestra.run.smithi150.stdout:1286 write 0x944a754 thru 0x9451981 (0x722e bytes) 2024-09-21T23:10:24.955 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:24.971 INFO:teuthology.orchestra.run.smithi150.stdout:1289 writesame 0xa774970 thru 0xa77d65e data_size 0x1(0x8cef bytes) 2024-09-21T23:10:24.971 INFO:teuthology.orchestra.run.smithi150.stdout:1290 read 0x3c21559 thru 0x3c25f66 (0x4a0e bytes) 2024-09-21T23:10:24.972 INFO:teuthology.orchestra.run.smithi150.stdout:1291 punch from 0x9df9015 to 0x9e006c8, (0x76b3 bytes) 2024-09-21T23:10:24.974 INFO:teuthology.orchestra.run.smithi150.stdout:1293 trunc from 0xa77d65f to 0x960b34c 2024-09-21T23:10:24.996 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:24.993+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:25.000 INFO:teuthology.orchestra.run.smithi150.stdout:1294 write 0x9f406d3 thru 0x9f4c40c (0xbd3a bytes) 2024-09-21T23:10:25.020 INFO:teuthology.orchestra.run.smithi150.stdout:1295 write 0xa38a9a0 thru 0xa38de1b (0x347c bytes) 2024-09-21T23:10:25.027 INFO:teuthology.orchestra.run.smithi150.stdout:1296 punch from 0x1ab7b2d to 0x1abb061, (0x3534 bytes) 2024-09-21T23:10:25.049 INFO:teuthology.orchestra.run.smithi150.stdout:1298 write 0xdf9fb57 thru 0xdfa4de9 (0x5293 bytes) 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.868086229999999, 2024-09-21T23:10:25.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.867983144, 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.057 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:10:25.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.867915795000002, 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.061 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:10:25.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.792728859, 2024-09-21T23:10:25.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:25.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.069 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.792673091000001, 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.070 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:10:25.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:10:25.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:10:25.073 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.792641551999999, 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.074 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:10:25.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.792608803, 2024-09-21T23:10:25.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:10:25.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:10:25.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.792581909999999, 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:10:25.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:10:25.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.792556637000001, 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:10:25.087 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:10:25.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:10:25.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.792525510000001, 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.092 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:10:25.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.791882154, 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.096 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.100 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786696227, 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.104 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786674743999999, 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.105 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:10:25.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:10:25.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:10:25.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786650458, 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786632790999999, 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.113 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:10:25.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:10:25.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786609206000001, 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.118 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:25.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:10:25.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:10:25.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786593191000001, 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:10:25.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786579360000001, 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.126 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:25.130 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786547999, 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.134 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.786479577000001, 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:25.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:25.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:25.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:25.139 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:25.139 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:25.140 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:25.137+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:25.141 INFO:teuthology.orchestra.run.smithi150.stdout:1300 write 0x1c7b965 thru 0x1c7df7e (0x261a bytes) 2024-09-21T23:10:25.141 INFO:teuthology.orchestra.run.smithi150.stdout:1301 write 0x8225973 thru 0x823214f (0xc7dd bytes) 2024-09-21T23:10:25.141 INFO:teuthology.orchestra.run.smithi150.stdout:1302 read 0x6b073c5 thru 0x6b1520b (0xde47 bytes) 2024-09-21T23:10:25.141 INFO:teuthology.orchestra.run.smithi150.stdout:1303 trunc from 0xdfa4dea to 0xa3783c6 2024-09-21T23:10:25.141 INFO:teuthology.orchestra.run.smithi150.stdout:1304 read 0x9b24b03 thru 0x9b29125 (0x4623 bytes) 2024-09-21T23:10:25.141 INFO:teuthology.orchestra.run.smithi150.stdout:1305 writesame 0xaffc31b thru 0xb00aed8 data_size 0x22(0xebbe bytes) 2024-09-21T23:10:25.141 INFO:teuthology.orchestra.run.smithi150.stdout:1306 writesame 0x8978300 thru 0x8978f73 data_size 0x2(0xc74 bytes) 2024-09-21T23:10:25.141 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:25.142 INFO:teuthology.orchestra.run.smithi150.stdout:1309 write 0xbe33a94 thru 0xbe43a25 (0xff92 bytes) 2024-09-21T23:10:25.154 INFO:teuthology.orchestra.run.smithi150.stdout:1310 writesame 0x3e6290e thru 0x3e674e9 data_size 0xa(0x4bdc bytes) 2024-09-21T23:10:25.161 INFO:teuthology.orchestra.run.smithi150.stdout:1311 write 0x7160c38 thru 0x716972b (0x8af4 bytes) 2024-09-21T23:10:25.185 INFO:teuthology.orchestra.run.smithi150.stdout:1312 write 0xebbd587 thru 0xebbfc8b (0x2705 bytes) 2024-09-21T23:10:25.190 INFO:teuthology.orchestra.run.smithi150.stdout:1313 write 0xcf96bde thru 0xcf97e18 (0x123b bytes) 2024-09-21T23:10:25.196 INFO:teuthology.orchestra.run.smithi150.stdout:1314 write 0x8293e34 thru 0x829581d (0x19ea bytes) 2024-09-21T23:10:25.202 INFO:teuthology.orchestra.run.smithi150.stdout:1315 writesame 0x63cb934 thru 0x63cc94c data_size 0x1(0x1019 bytes) 2024-09-21T23:10:25.211 INFO:teuthology.orchestra.run.smithi150.stdout:1316 punch from 0x231ea32 to 0x23223d0, (0x399e bytes) 2024-09-21T23:10:25.211 INFO:teuthology.orchestra.run.smithi150.stdout:1317 writesame 0x1d1063b thru 0x1d157df data_size 0x1(0x51a5 bytes) 2024-09-21T23:10:25.231 INFO:teuthology.orchestra.run.smithi150.stdout:1318 writesame 0x9a60d6c thru 0x9a6efe7 data_size 0x41(0xe27c bytes) 2024-09-21T23:10:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:1321 writesame 0xc38c226 thru 0xc38dc83 data_size 0x2d(0x1a5e bytes) 2024-09-21T23:10:25.240 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:25.237+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:25.246 INFO:teuthology.orchestra.run.smithi150.stdout:1322 write 0x3a78fd3 thru 0x3a7bb02 (0x2b30 bytes) 2024-09-21T23:10:25.251 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:25.251 INFO:teuthology.orchestra.run.smithi150.stdout:1326 write 0x691210 thru 0x69a1fd (0x8fee bytes) 2024-09-21T23:10:25.265 INFO:teuthology.orchestra.run.smithi150.stdout:1327 read 0xa0bab47 thru 0xa0c409d (0x9557 bytes) 2024-09-21T23:10:25.265 INFO:teuthology.orchestra.run.smithi150.stdout:1328 punch from 0x418784b to 0x4194dd3, (0xd588 bytes) 2024-09-21T23:10:25.265 INFO:teuthology.orchestra.run.smithi150.stdout:1329 read 0x72e6e55 thru 0x72ea864 (0x3a10 bytes) 2024-09-21T23:10:25.265 INFO:teuthology.orchestra.run.smithi150.stdout:1331 read 0x736c690 thru 0x737613b (0x9aac bytes) 2024-09-21T23:10:25.265 INFO:teuthology.orchestra.run.smithi150.stdout:1332 read 0x9c0e840 thru 0x9c156a6 (0x6e67 bytes) 2024-09-21T23:10:25.265 INFO:teuthology.orchestra.run.smithi150.stdout:1333 write 0xa4542cf thru 0xa463b28 (0xf85a bytes) 2024-09-21T23:10:25.275 INFO:teuthology.orchestra.run.smithi150.stdout:1334 read 0x437c2a0 thru 0x4386fb7 (0xad18 bytes) 2024-09-21T23:10:25.275 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:25.275 INFO:teuthology.orchestra.run.smithi150.stdout:1336 writesame 0x23eeb29 thru 0x23f87b6 data_size 0x1d(0x9c8e bytes) 2024-09-21T23:10:25.279 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:25.279 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:25.279 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:25.279 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:25.287 INFO:teuthology.orchestra.run.smithi150.stdout:1337 punch from 0xd567afc to 0xd56a9e5, (0x2ee9 bytes) 2024-09-21T23:10:25.287 INFO:teuthology.orchestra.run.smithi150.stdout:1339 write 0x392aaae thru 0x392f82b (0x4d7e bytes) 2024-09-21T23:10:25.294 INFO:teuthology.orchestra.run.smithi150.stdout:1340 write 0x85443cd thru 0x854fadd (0xb711 bytes) 2024-09-21T23:10:25.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:25.300 INFO:teuthology.orchestra.run.smithi150.stdout:1341 read 0x1287a67 thru 0x128895b (0xef5 bytes) 2024-09-21T23:10:25.302 INFO:teuthology.orchestra.run.smithi150.stdout:1343 writesame 0xc12088e thru 0xc125d00 data_size 0x1(0x5473 bytes) 2024-09-21T23:10:25.312 INFO:teuthology.orchestra.run.smithi150.stdout:1344 write 0x61d5768 thru 0x61d7d86 (0x261f bytes) 2024-09-21T23:10:25.320 INFO:teuthology.orchestra.run.smithi150.stdout:1345 punch from 0xb67591 to 0xb6c514, (0x4f83 bytes) 2024-09-21T23:10:25.320 INFO:teuthology.orchestra.run.smithi150.stdout:1348 read 0x96f6edf thru 0x96fbe07 (0x4f29 bytes) 2024-09-21T23:10:25.320 INFO:teuthology.orchestra.run.smithi150.stdout:1350 write 0x215e248 thru 0x216cd0c (0xeac5 bytes) 2024-09-21T23:10:25.331 INFO:teuthology.orchestra.run.smithi150.stdout:1351 write 0xde14e13 thru 0xde23da5 (0xef93 bytes) 2024-09-21T23:10:25.338 INFO:teuthology.orchestra.run.smithi150.stdout:1352 punch from 0xb3d337b to 0xb3d99b7, (0x663c bytes) 2024-09-21T23:10:25.338 INFO:teuthology.orchestra.run.smithi150.stdout:1353 writesame 0x796035b thru 0x7962570 data_size 0x1(0x2216 bytes) 2024-09-21T23:10:25.340 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:25.337+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:25.369 INFO:teuthology.orchestra.run.smithi150.stdout:1355 trunc from 0xebbfc8c to 0x21e1c24 2024-09-21T23:10:25.419 INFO:teuthology.orchestra.run.smithi150.stdout:1357 punch from 0x15c5475 to 0x15d2a19, (0xd5a4 bytes) 2024-09-21T23:10:25.419 INFO:teuthology.orchestra.run.smithi150.stdout:1358 punch from 0x1310e73 to 0x1318fc0, (0x814d bytes) 2024-09-21T23:10:25.419 INFO:teuthology.orchestra.run.smithi150.stdout:1359 read 0x1c07137 thru 0x1c0d81e (0x66e8 bytes) 2024-09-21T23:10:25.420 INFO:teuthology.orchestra.run.smithi150.stdout:1360 writesame 0x165efda thru 0x166d01d data_size 0x1f(0xe044 bytes) 2024-09-21T23:10:25.433 INFO:teuthology.orchestra.run.smithi150.stdout:1363 read 0x2a2ca4 thru 0x2ae412 (0xb76f bytes) 2024-09-21T23:10:25.437 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:25.437 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:25.438 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:25.438 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:25.438 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:25.442 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:25.441+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:25.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:25.466 INFO:teuthology.orchestra.run.smithi150.stdout:1364 writesame 0xd91dff8 thru 0xd921d3f data_size 0x4a(0x3d48 bytes) 2024-09-21T23:10:25.482 INFO:teuthology.orchestra.run.smithi150.stdout:1365 trunc from 0xd921d40 to 0x477dfce 2024-09-21T23:10:25.490 INFO:teuthology.orchestra.run.smithi150.stdout:1366 read 0x2389db5 thru 0x2393990 (0x9bdc bytes) 2024-09-21T23:10:25.492 INFO:teuthology.orchestra.run.smithi150.stdout:1367 trunc from 0x477dfce to 0x3885f28 2024-09-21T23:10:25.499 INFO:teuthology.orchestra.run.smithi150.stdout:1368 punch from 0x3181966 to 0x31882af, (0x6949 bytes) 2024-09-21T23:10:25.500 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:25.515 INFO:teuthology.orchestra.run.smithi150.stdout:1370 trunc from 0x3885f28 to 0xd0fd06d 2024-09-21T23:10:25.524 INFO:teuthology.orchestra.run.smithi150.stdout:1371 trunc from 0xd0fd06d to 0xbed9b1b 2024-09-21T23:10:25.543 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:25.537+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:25.544 INFO:teuthology.orchestra.run.smithi150.stdout:1372 writesame 0xe28b9c7 thru 0xe29b0f0 data_size 0x11(0xf72a bytes) 2024-09-21T23:10:25.544 INFO:teuthology.orchestra.run.smithi150.stdout:1373 writesame 0xac3bdd5 thru 0xac49fb7 data_size 0xb(0xe1e3 bytes) 2024-09-21T23:10:25.554 INFO:teuthology.orchestra.run.smithi150.stdout:1374 read 0x6e1226 thru 0x6f0f3b (0xfd16 bytes) 2024-09-21T23:10:25.555 INFO:teuthology.orchestra.run.smithi150.stdout:1375 read 0x766ca1a thru 0x7673d6c (0x7353 bytes) 2024-09-21T23:10:25.555 INFO:teuthology.orchestra.run.smithi150.stdout:1376 write 0xadb48d6 thru 0xadc1318 (0xca43 bytes) 2024-09-21T23:10:25.564 INFO:teuthology.orchestra.run.smithi150.stdout:1377 read 0x31e305c thru 0x31e8f19 (0x5ebe bytes) 2024-09-21T23:10:25.564 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:25.571 INFO:teuthology.orchestra.run.smithi150.stdout:1379 trunc from 0xe29b0f1 to 0xa1b98e9 2024-09-21T23:10:25.590 INFO:teuthology.orchestra.run.smithi150.stdout:1380 trunc from 0xa1b98e9 to 0x97d07c1 2024-09-21T23:10:25.600 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12737 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2043904~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500882419, 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035132690000000001, 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12737 2024-09-21T23:10:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097394+0000", 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097397+0000", 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9890000000000001e-06 2024-09-21T23:10:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097400+0000", 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132379+0000", 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034979237000000003 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132438+0000", 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8183999999999997e-05 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132526+0000", 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8171999999999995e-05 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12738 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2058240~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500872132, 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035265405999999999, 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12738 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097404+0000", 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097405+0000", 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097408+0000", 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.971e-06 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097410+0000", 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132558+0000", 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035147993000000002 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132611+0000", 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2293000000000002e-05 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132669+0000", 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8139e-05 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12739 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2072576~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500862066, 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035408621000000001, 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12739 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097414+0000", 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097415+0000", 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.046e-06 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097418+0000", 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9629999999999998e-06 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097424+0000", 2024-09-21T23:10:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.891e-06 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132706+0000", 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035282154000000003 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132762+0000", 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6475999999999999e-05 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132822+0000", 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9877999999999998e-05 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12740 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2086912~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500846001999999, 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035536514999999998, 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12740 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6399999999999998e-07 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097432+0000", 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.889e-06 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097438+0000", 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9270000000000002e-06 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097442+0000", 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4340000000000002e-06 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132856+0000", 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035413922 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132906+0000", 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9997999999999998e-05 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132966+0000", 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9981000000000001e-05 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12797 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1313792~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500262411, 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034983512000000001, 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12797 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098015+0000", 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1850000000000001e-06 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098018+0000", 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1540000000000002e-06 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098020+0000", 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6390000000000001e-06 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132848+0000", 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034828143999999998 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132909+0000", 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0825999999999997e-05 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132997+0000", 2024-09-21T23:10:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7348999999999993e-05 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12798 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1328128~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500251813, 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035158699000000002, 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12798 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098025+0000", 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0440000000000001e-06 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098028+0000", 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098031+0000", 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7499999999999999e-06 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133037+0000", 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035005938 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133095+0000", 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8208000000000002e-05 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133182+0000", 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7559000000000007e-05 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12799 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1342464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500241178, 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035342140000000001, 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12799 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098035+0000", 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098036+0000", 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.172e-06 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098040+0000", 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8319999999999999e-06 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098042+0000", 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8509999999999999e-06 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133220+0000", 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035177096999999997 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133288+0000", 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8534000000000005e-05 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133376+0000", 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8441000000000002e-05 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12800 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1356800~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500230062, 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03547894, 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12800 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098046+0000", 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098047+0000", 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098051+0000", 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.687e-06 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098053+0000", 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.689e-06 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133401+0000", 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035347575999999999 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133446+0000", 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4962000000000001e-05 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133524+0000", 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8568999999999999e-05 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12801 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1371136~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500218801999999, 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035624792000000002, 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12801 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098058+0000", 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.105e-06 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098062+0000", 2024-09-21T23:10:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.844e-06 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098065+0000", 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7310000000000002e-06 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133557+0000", 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035492255 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133605+0000", 2024-09-21T23:10:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8569000000000002e-05 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133682+0000", 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6070999999999997e-05 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12802 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1385472~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500207674, 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035748412, 2024-09-21T23:10:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12802 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098069+0000", 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:10:25.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098072+0000", 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.033e-06 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098075+0000", 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8490000000000002e-06 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133709+0000", 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035633962999999998 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133753+0000", 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3809e-05 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133816+0000", 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3301999999999998e-05 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12803 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1399808~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500196576, 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035862430000000001, 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12803 2024-09-21T23:10:25.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:25.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098080+0000", 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098083+0000", 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0479999999999999e-06 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098087+0000", 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2030000000000001e-06 2024-09-21T23:10:25.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133842+0000", 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035755012000000003 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133885+0000", 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000002e-05 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133941+0000", 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6648999999999999e-05 2024-09-21T23:10:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12804 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1414144~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500185857, 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035982214999999998, 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12804 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098091+0000", 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2449999999999999e-06 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098094+0000", 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0120000000000001e-06 2024-09-21T23:10:25.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098097+0000", 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7199999999999998e-06 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133968+0000", 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035871003999999998 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134016+0000", 2024-09-21T23:10:25.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8368999999999997e-05 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134072+0000", 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5649999999999997e-05 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12805 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1428480~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500175659, 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036102355000000003, 2024-09-21T23:10:25.653 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12805 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098101+0000", 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-21T23:10:25.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098110+0000", 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1290000000000002e-06 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098113+0000", 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8310000000000001e-06 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134100+0000", 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035986481000000001 2024-09-21T23:10:25.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134144+0000", 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3890000000000002e-05 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134202+0000", 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8613000000000003e-05 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12806 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1442816~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.657 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500159335999999, 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036273185999999999, 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12806 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:25.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098117+0000", 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098118+0000", 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.124e-06 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098121+0000", 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098124+0000", 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8320000000000002e-06 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134229+0000", 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036105406 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134295+0000", 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6282e-05 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134389+0000", 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4168999999999999e-05 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12807 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1457152~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500147046, 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037538491, 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12807 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098129+0000", 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2100000000000001e-07 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098130+0000", 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1149999999999999e-06 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098133+0000", 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.258e-06 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098136+0000", 2024-09-21T23:10:25.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7810000000000001e-06 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135467+0000", 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037331371000000002 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135549+0000", 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2211000000000003e-05 2024-09-21T23:10:25.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135667+0000", 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117534 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12808 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1471488~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500136519, 2024-09-21T23:10:25.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037746517, 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12808 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098142+0000", 2024-09-21T23:10:25.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098145+0000", 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098148+0000", 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7750000000000001e-06 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135714+0000", 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037566612999999999 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135788+0000", 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4000999999999998e-05 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135886+0000", 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7272e-05 2024-09-21T23:10:25.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12809 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1485824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500124766000001, 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037999241000000003, 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12809 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1e-07 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098152+0000", 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0130000000000001e-06 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098155+0000", 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0290000000000001e-06 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098158+0000", 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.565e-06 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135936+0000", 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037778467000000003 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136047+0000", 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011086099999999999 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136150+0000", 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103096 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12810 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1500160~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500115005, 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038237449, 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.676 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12810 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098162+0000", 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.189e-06 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098165+0000", 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9780000000000001e-06 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098168+0000", 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136187+0000", 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038019325999999999 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136272+0000", 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4497000000000001e-05 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136398+0000", 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012611600000000001 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.680 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12811 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1514496~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.500104538, 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038486382, 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12811 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098172+0000", 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.24e-06 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098175+0000", 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.976e-06 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098178+0000", 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8200000000000001e-06 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136448+0000", 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038269965000000003 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136543+0000", 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5069999999999996e-05 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136657+0000", 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114071 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21978 3.1f 3:fa161337:::rbd_data.1149c115bf92.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.8649723110000007, 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052821716999999997, 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21978 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:25.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733307+0000", 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:25.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733321+0000", 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4049999999999999e-05 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733357+0000", 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6363999999999999e-05 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733375+0000", 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733493+0000", 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117642 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733629+0000", 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000136596 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.742696+0000", 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090670260000000006 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.785924+0000", 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043228124 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.785957+0000", 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2252999999999998e-05 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.786048+0000", 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1731999999999995e-05 2024-09-21T23:10:25.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.786125+0000", 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6606999999999993e-05 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:25.690 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:25.691 INFO:teuthology.orchestra.run.smithi150.stdout:1382 writesame 0x911b082 thru 0x912367b data_size 0xb(0x85fa bytes) 2024-09-21T23:10:25.691 INFO:teuthology.orchestra.run.smithi150.stdout:1383 read 0x1233d37 thru 0x1242d98 (0xf062 bytes) 2024-09-21T23:10:25.691 INFO:teuthology.orchestra.run.smithi150.stdout:1384 write 0x48bb02b thru 0x48be4f8 (0x34ce bytes) 2024-09-21T23:10:25.692 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:25.693 INFO:teuthology.orchestra.run.smithi150.stdout:1385 punch from 0x5bb8807 to 0x5bc2e56, (0xa64f bytes) 2024-09-21T23:10:25.693 INFO:teuthology.orchestra.run.smithi150.stdout:1386 write 0xd7e113 thru 0xd8b084 (0xcf72 bytes) 2024-09-21T23:10:25.693 INFO:teuthology.orchestra.run.smithi150.stdout:1388 punch from 0x47ba459 to 0x47c93e7, (0xef8e bytes) 2024-09-21T23:10:25.693 INFO:teuthology.orchestra.run.smithi150.stdout:1389 punch from 0x1eba57 to 0x1f2934, (0x6edd bytes) 2024-09-21T23:10:25.693 INFO:teuthology.orchestra.run.smithi150.stdout:1390 trunc from 0x97d07c1 to 0xb789aa9 2024-09-21T23:10:25.693 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:25.693 INFO:teuthology.orchestra.run.smithi150.stdout:1392 clone 16 order 21 su 32768 sc 7 2024-09-21T23:10:25.694 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:25.693+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:25.795 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:25.793+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:25.864 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:25.864 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:25.864 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:25.878 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:25.897 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:25.897+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:25.997 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:25.993+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:26.000 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:26.000 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:26.000 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:26.000 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:26.000 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:26.014 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:26.099 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:26.097+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:26.184 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4287 3.0 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1301057~9663] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.025674+0000", 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.156737009, 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019368100999999999, 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4287 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.025674+0000", 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.025674+0000", 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.025676+0000", 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027586+0000", 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019099589999999999 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027620+0000", 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3729000000000002e-05 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027635+0000", 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5024999999999999e-05 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027749+0000", 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114707 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027934+0000", 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00018491300000000001 2024-09-21T23:10:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.028951+0000", 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00101639 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044295+0000", 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01534408 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044933+0000", 2024-09-21T23:10:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00063873300000000005 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044992+0000", 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8445e-05 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.045042+0000", 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130999999999998e-05 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4886 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2949120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.181352+0000", 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.001058689000001, 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01390837, 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4886 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181352+0000", 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181352+0000", 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181353+0000", 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7199999999999998e-07 2024-09-21T23:10:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181357+0000", 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.676e-06 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181371+0000", 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4139e-05 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181380+0000", 2024-09-21T23:10:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2520000000000005e-06 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.194935+0000", 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013555322999999999 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195123+0000", 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187982 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195261+0000", 2024-09-21T23:10:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000137226 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4887 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.001019366000001, 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014700754, 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4887 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181396+0000", 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.67e-06 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181407+0000", 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181418+0000", 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1088e-05 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195304+0000", 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013885817999999999 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195756+0000", 2024-09-21T23:10:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00045195899999999999 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196092+0000", 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000336201 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4888 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3866624~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.00098023, 2024-09-21T23:10:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015118104, 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4888 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181442+0000", 2024-09-21T23:10:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.024e-05 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181454+0000", 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2788e-05 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181464+0000", 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.876e-06 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196122+0000", 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014657701 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196192+0000", 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9920999999999997e-05 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196549+0000", 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035691199999999997 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5553 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.983818115999998, 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013603697999999999, 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5553 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7700000000000004e-07 2024-09-21T23:10:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198596+0000", 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5270000000000001e-06 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198605+0000", 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2089999999999994e-06 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198612+0000", 2024-09-21T23:10:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8079999999999999e-06 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.211710+0000", 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013098205 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212064+0000", 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035353899999999999 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212197+0000", 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013283300000000001 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5554 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198620+0000", 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.983790459000002, 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0138052, 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5554 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198620+0000", 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198620+0000", 2024-09-21T23:10:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198621+0000", 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8100000000000003e-07 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198625+0000", 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.602e-06 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198633+0000", 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8210000000000008e-06 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198640+0000", 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7009999999999999e-06 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212306+0000", 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013665773000000001 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212365+0000", 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8752000000000003e-05 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212426+0000", 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000001e-05 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5555 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.983762055, 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014241567, 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5555 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2099999999999997e-07 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198652+0000", 2024-09-21T23:10:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6620000000000001e-06 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198660+0000", 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2449999999999998e-06 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198667+0000", 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.737e-06 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212444+0000", 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013776563 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212576+0000", 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013281 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212890+0000", 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031402900000000001 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20822 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.489864431999999, 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015204284, 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20822 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8900000000000001e-07 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692549+0000", 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8309999999999999e-06 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692552+0000", 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7100000000000001e-06 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692555+0000", 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.084e-06 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.703174+0000", 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010618588 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.703295+0000", 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120871 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.703583+0000", 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028789400000000002 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707646+0000", 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040630279999999998 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707674+0000", 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8320999999999999e-05 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707707+0000", 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3197000000000001e-05 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707751+0000", 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3470999999999998e-05 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20823 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.489851821, 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015300924, 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.226 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20823 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692561+0000", 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8509999999999999e-06 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692565+0000", 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6600000000000001e-06 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692569+0000", 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.704019+0000", 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011450435 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.704079+0000", 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9221000000000001e-05 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.704345+0000", 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026609600000000001 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707764+0000", 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034193470000000001 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707792+0000", 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8053000000000001e-05 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707822+0000", 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0219999999999999e-05 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707860+0000", 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7808999999999997e-05 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.231 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20824 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692581+0000", 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.489829706, 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015439424, 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20824 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692581+0000", 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692581+0000", 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692582+0000", 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0699999999999998e-07 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692590+0000", 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4500000000000004e-06 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692595+0000", 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2170000000000002e-06 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692599+0000", 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.788e-06 2024-09-21T23:10:26.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.705269+0000", 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012669654000000001 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.705424+0000", 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000155067 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.705963+0000", 2024-09-21T23:10:26.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00053946500000000004 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707767+0000", 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00180382 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707927+0000", 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016015800000000001 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707989+0000", 2024-09-21T23:10:26.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1366999999999997e-05 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708021+0000", 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2030999999999999e-05 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20825 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692604+0000", 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.489806571999999, 2024-09-21T23:10:26.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016684879999999999, 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20825 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692604+0000", 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692604+0000", 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692605+0000", 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8299999999999998e-07 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692607+0000", 2024-09-21T23:10:26.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1629999999999999e-06 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692611+0000", 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9249999999999997e-06 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692614+0000", 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.455e-06 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.706708+0000", 2024-09-21T23:10:26.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014093879 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.706822+0000", 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113947 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707231+0000", 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040900499999999999 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709009+0000", 2024-09-21T23:10:26.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017782900000000001 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709061+0000", 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1283999999999999e-05 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709150+0000", 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9431999999999993e-05 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709289+0000", 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013921699999999999 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20826 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.489792992, 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017943988000000001, 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20826 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:26.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4699999999999998e-07 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692620+0000", 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9360000000000002e-06 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692633+0000", 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3106e-05 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692638+0000", 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7700000000000001e-06 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708072+0000", 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015434045 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708175+0000", 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102643 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708530+0000", 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355688 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709445+0000", 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000914655 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.710359+0000", 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00091388999999999999 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.710477+0000", 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000118424 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.710562+0000", 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4584000000000002e-05 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.248 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21677 3.1a 3:5a717256:::rbd_data.1149f84cbfe6.0000000000000010:c [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.331989+0000", 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8504220750000009, 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013983705000000001, 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.249 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21677 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331989+0000", 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331989+0000", 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331990+0000", 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.35e-06 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331998+0000", 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5669999999999997e-06 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332019+0000", 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1613000000000001e-05 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332034+0000", 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4276e-05 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.345477+0000", 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01344365 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.345697+0000", 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022008800000000001 2024-09-21T23:10:26.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.345973+0000", 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000275161 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21678 3.1a 3:5a717256:::rbd_data.1149f84cbfe6.0000000000000010:c [read 187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.332056+0000", 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8503550359999998, 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014483438, 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.253 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21678 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332056+0000", 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332056+0000", 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332058+0000", 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6029999999999999e-06 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332064+0000", 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7070000000000003e-06 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332082+0000", 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8117e-05 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332097+0000", 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4759999999999999e-05 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346050+0000", 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013953402 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346266+0000", 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215038 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346539+0000", 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273811 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.257 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21679 3.1a 3:5a717256:::rbd_data.1149f84cbfe6.0000000000000010:c [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.332116+0000", 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8502946589999993, 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014976817, 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21679 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332116+0000", 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332116+0000", 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332117+0000", 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-21T23:10:26.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332123+0000", 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332140+0000", 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7563000000000001e-05 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332154+0000", 2024-09-21T23:10:26.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3692999999999999e-05 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346617+0000", 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014463055000000001 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346826+0000", 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000208975 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347093+0000", 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026689800000000001 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21705 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 18432~18432] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.332944+0000", 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8494666300000002, 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014791653, 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21705 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332944+0000", 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332944+0000", 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332947+0000", 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2900000000000001e-06 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332958+0000", 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.169e-05 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332989+0000", 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0904000000000001e-05 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333002+0000", 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.309e-05 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347166+0000", 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014164177 2024-09-21T23:10:26.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347409+0000", 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024228499999999999 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347736+0000", 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032721699999999998 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21706 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333046+0000", 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8493651230000001, 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015328334000000001, 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21706 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333046+0000", 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333046+0000", 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333047+0000", 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-06 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333054+0000", 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333075+0000", 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0636e-05 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333098+0000", 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2937000000000001e-05 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347820+0000", 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014721916 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348035+0000", 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021504200000000001 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348374+0000", 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033955999999999998 2024-09-21T23:10:26.270 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21707 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333138+0000", 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8492729009999991, 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015804684999999999, 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21707 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333138+0000", 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333138+0000", 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333140+0000", 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4640000000000001e-06 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333145+0000", 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8760000000000002e-06 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333172+0000", 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.654e-05 2024-09-21T23:10:26.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333186+0000", 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.413e-05 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348451+0000", 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015265435000000001 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348670+0000", 2024-09-21T23:10:26.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000218805 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348943+0000", 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027243500000000002 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21708 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333206+0000", 2024-09-21T23:10:26.275 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8492049149999996, 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016345584, 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21708 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333206+0000", 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333206+0000", 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333207+0000", 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333213+0000", 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8170000000000004e-06 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333233+0000", 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9729999999999999e-05 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333246+0000", 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3603e-05 2024-09-21T23:10:26.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349021+0000", 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015774921000000001 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349272+0000", 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025024500000000001 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349552+0000", 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000280098 2024-09-21T23:10:26.279 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21709 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333265+0000", 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8491457520000001, 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016949235999999999, 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.280 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21709 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333265+0000", 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333265+0000", 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333266+0000", 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.15e-06 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333272+0000", 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5740000000000003e-06 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333291+0000", 2024-09-21T23:10:26.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9409e-05 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333304+0000", 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2984e-05 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349628+0000", 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016324113000000001 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349920+0000", 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029203500000000001 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.350214+0000", 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029397100000000002 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.284 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:26.286 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:26.286 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:26.285+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:26.387 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:26.381+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:26.456 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:26.456 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:26.456 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:26.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:26.512 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:26.509+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:26.588 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:26.585+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:26.640 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:26.641 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:26.641 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:26.641 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:26.641 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:26.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:10:26.657 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:26.699 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:26.697+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:26.789 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:26.785+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:26.823 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:26.823 INFO:teuthology.orchestra.run.smithi150.stdout:leaving image image_client.0-clone15 intact 2024-09-21T23:10:26.846 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:26.846 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.130166838999997, 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039594890000000001, 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716126+0000", 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:26.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716131+0000", 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716144+0000", 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2267e-05 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716158+0000", 2024-09-21T23:10:26.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4731e-05 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716239+0000", 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1013999999999994e-05 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716271+0000", 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1535999999999998e-05 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720016+0000", 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037447399999999999 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720084+0000", 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8813999999999995e-05 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.851 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2850 3.4 e30/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 30'30, mlcod=30'28)", 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.820275416000001, 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033350860000000001, 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026017+0000", 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.68e-07 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026023+0000", 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1959999999999996e-06 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026033+0000", 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8120000000000002e-06 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026042+0000", 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7059999999999993e-06 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026127+0000", 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4973000000000001e-05 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026145+0000", 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8422000000000001e-05 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029303+0000", 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031578980000000001 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029351+0000", 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8411000000000001e-05 2024-09-21T23:10:26.855 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:4287 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'16, mlcod=31'14)", 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.816923404000001, 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015395872, 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029371+0000", 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2759999999999999e-06 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031303+0000", 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019322359999999999 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031331+0000", 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8408999999999999e-05 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031342+0000", 2024-09-21T23:10:26.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1086e-05 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031428+0000", 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5990999999999997e-05 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031454+0000", 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044695+0000", 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013240784 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044764+0000", 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9623999999999996e-05 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6012 3.4 e31/21 3:21794b57:::rbd_data.1149d3a069d9.0000000000000003:head v 31'34, mlcod=30'31)", 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.577900167999999, 2024-09-21T23:10:26.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094147550000000003, 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268393+0000", 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.72e-06 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269210+0000", 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081656700000000003 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269235+0000", 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5000000000000001e-05 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269248+0000", 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3393e-05 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269314+0000", 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6180000000000007e-05 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269341+0000", 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6554999999999999e-05 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277723+0000", 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083823960000000003 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277806+0000", 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2943999999999997e-05 2024-09-21T23:10:26.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6173 3.0 e32/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 32'22, mlcod=31'19)", 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.247858281999999, 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002753003, 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598435+0000", 2024-09-21T23:10:26.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8870000000000001e-06 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598445+0000", 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4949999999999997e-06 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598464+0000", 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8720999999999999e-05 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598473+0000", 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5980000000000003e-06 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598540+0000", 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7096999999999998e-05 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598564+0000", 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3288000000000001e-05 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601136+0000", 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025726099999999999 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601186+0000", 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0306999999999998e-05 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11716 3.9 3:94516a46:::rbd_data.1149e13ad964.000000000000006f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:26.869 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.916458942, 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097894799999999997, 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11716 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:26.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929835+0000", 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2529999999999999e-06 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929846+0000", 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0859000000000001e-05 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929869+0000", 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3450000000000001e-05 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929879+0000", 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7270000000000002e-06 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929973+0000", 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3917000000000002e-05 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.930106+0000", 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013346199999999999 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939479+0000", 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.009372755 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939522+0000", 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3025999999999998e-05 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939622+0000", 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010003100000000001 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:12080 3.0 e36/21 3:003353e5:::rbd_data.1149e13ad964.000000000000003f:head v 36'38, mlcod=35'35)", 2024-09-21T23:10:26.874 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.259204619, 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045448859999999997, 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587090+0000", 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.367e-06 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587109+0000", 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9816000000000001e-05 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587158+0000", 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8513999999999997e-05 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587179+0000", 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587395+0000", 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021613099999999999 2024-09-21T23:10:26.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587449+0000", 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3316000000000002e-05 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.591513+0000", 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040644160000000004 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.591632+0000", 2024-09-21T23:10:26.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011884599999999999 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21535 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.125532+0000", 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.720759605, 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022049029999999998, 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.879 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21535 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125532+0000", 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125532+0000", 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125535+0000", 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125551+0000", 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5483000000000001e-05 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125585+0000", 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4489999999999997e-05 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125602+0000", 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6529999999999999e-05 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125744+0000", 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014231199999999999 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125909+0000", 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016517800000000001 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.127518+0000", 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016084719999999999 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.127615+0000", 2024-09-21T23:10:26.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7134999999999997e-05 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.127737+0000", 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122099 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21566 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.58097042, 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002480384, 2024-09-21T23:10:26.884 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21566 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265324+0000", 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9500000000000001e-06 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265341+0000", 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6303e-05 2024-09-21T23:10:26.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265373+0000", 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2427e-05 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265408+0000", 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5126000000000002e-05 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265555+0000", 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014723500000000001 2024-09-21T23:10:26.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265818+0000", 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026269800000000001 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267587+0000", 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017692000000000001 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267679+0000", 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1922999999999994e-05 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267802+0000", 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012252199999999999 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21574 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.56028618, 2024-09-21T23:10:26.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024147919999999998, 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21574 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286008+0000", 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286021+0000", 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2714000000000001e-05 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286051+0000", 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0633000000000003e-05 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286082+0000", 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0213e-05 2024-09-21T23:10:26.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286304+0000", 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000222112 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286570+0000", 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026670399999999999 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288119+0000", 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015483579999999999 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288208+0000", 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9133000000000002e-05 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288420+0000", 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021242900000000001 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21904 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.556415+0000", 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.289876766999999, 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002137024, 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21904 2024-09-21T23:10:26.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556415+0000", 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556415+0000", 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556418+0000", 2024-09-21T23:10:26.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.19e-06 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556433+0000", 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5119999999999999e-05 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556470+0000", 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6242999999999998e-05 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556487+0000", 2024-09-21T23:10:26.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7533e-05 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556612+0000", 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012447900000000001 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.556759+0000", 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000147899 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.558143+0000", 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013831589999999999 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.558303+0000", 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016023599999999999 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.558552+0000", 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024916499999999998 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21988 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.012013165000001, 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024888240000000002, 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.900 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21988 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834281+0000", 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-06 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834294+0000", 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2306e-05 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834323+0000", 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.885e-05 2024-09-21T23:10:26.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834337+0000", 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4735999999999999e-05 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834488+0000", 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015039599999999999 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834745+0000", 2024-09-21T23:10:26.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000257385 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836546+0000", 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018004220000000001 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836638+0000", 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2483000000000002e-05 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836767+0000", 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012939900000000001 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:22023 3.0 e37/21 3:020038fe:::rbd_data.1149c115bf92.0000000000000003:head v 37'57, mlcod=37'55)", 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.931245+0000", 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.9150467340000006, 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023235590000000002, 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.905 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931245+0000", 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931245+0000", 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931247+0000", 2024-09-21T23:10:26.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0640000000000001e-06 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931341+0000", 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3795999999999994e-05 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931373+0000", 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1665999999999998e-05 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931386+0000", 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3482999999999999e-05 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931522+0000", 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013622600000000001 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931574+0000", 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2080000000000003e-05 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.933482+0000", 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019074280000000001 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.933569+0000", 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6816000000000004e-05 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:22077 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.7788748000000005, 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025167950000000001, 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.910 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 22077 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067420+0000", 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7829999999999999e-06 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067434+0000", 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4116e-05 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067467+0000", 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2790999999999998e-05 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067489+0000", 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067656+0000", 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016664100000000001 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067913+0000", 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000257154 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069732+0000", 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001819036 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069820+0000", 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7590000000000007e-05 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069934+0000", 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011401200000000001 2024-09-21T23:10:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:22098 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.2698738879999993, 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066791929999999999, 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 22098 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576421+0000", 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6090000000000001e-06 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576439+0000", 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7802e-05 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576481+0000", 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1653999999999998e-05 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576507+0000", 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6001000000000001e-05 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576679+0000", 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017189700000000001 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576707+0000", 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8608000000000001e-05 2024-09-21T23:10:26.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.580886+0000", 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041785920000000001 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.581057+0000", 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000170915 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.582894+0000", 2024-09-21T23:10:26.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018374070000000001 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.582981+0000", 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6436999999999999e-05 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.583097+0000", 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116271 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29041 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.8830348080000001, 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024231119999999998, 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29041 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963260+0000", 2024-09-21T23:10:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.224e-06 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963272+0000", 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1817999999999999e-05 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963301+0000", 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9268000000000002e-05 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963315+0000", 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3756e-05 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963483+0000", 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167863 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963752+0000", 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026917 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965465+0000", 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001713259 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965559+0000", 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3521000000000001e-05 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965680+0000", 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121233 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29116 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.7457176620000001, 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024782770000000001, 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29116 2024-09-21T23:10:26.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100577+0000", 2024-09-21T23:10:26.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100590+0000", 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2959000000000001e-05 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100620+0000", 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9833e-05 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100634+0000", 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4167e-05 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100794+0000", 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015995500000000001 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.101056+0000", 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026250900000000002 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.102829+0000", 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001772259 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.102924+0000", 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4874e-05 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.103052+0000", 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000128856 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29126 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.115287+0000", 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.7310050160000001, 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002371293, 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29126 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115287+0000", 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115287+0000", 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115290+0000", 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0989999999999999e-06 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115302+0000", 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.198e-05 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115332+0000", 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0558000000000001e-05 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115346+0000", 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3925e-05 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115507+0000", 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016117099999999999 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115776+0000", 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026852700000000001 2024-09-21T23:10:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.117491+0000", 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001715428 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.117559+0000", 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8065999999999995e-05 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.117658+0000", 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539e-05 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29244 3.9 3:95c8871e:::rbd_data.11499299167c.0000000000000302:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.539688562, 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080285210000000003, 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29244 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306606+0000", 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306620+0000", 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3978999999999999e-05 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306642+0000", 2024-09-21T23:10:26.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2557000000000001e-05 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306655+0000", 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2894000000000001e-05 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306715+0000", 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9485999999999999e-05 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306804+0000", 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9603000000000003e-05 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314472+0000", 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076677029999999997 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314552+0000", 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0275e-05 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314632+0000", 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9599999999999997e-05 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29590 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.219022508, 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058105209999999999, 2024-09-21T23:10:26.941 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29590 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627273+0000", 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7069999999999998e-06 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627288+0000", 2024-09-21T23:10:26.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5129e-05 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627322+0000", 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3494999999999997e-05 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627347+0000", 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5191e-05 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627458+0000", 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011141000000000001 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627467+0000", 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.630921+0000", 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00345467 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.631080+0000", 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015890500000000001 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.632864+0000", 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017839729999999999 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.632961+0000", 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6784e-05 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:26.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.633080+0000", 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001189 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:26.947 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:26.948 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:26.949 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:26.945+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:26.965 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:26.965 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:26.983 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:26.984 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:27.004 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:27.024 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:27.024 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:27.024 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:27.045 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:27.045 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:27.049 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:27.045+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:27.069 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:27.069 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:27.088 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:27.088 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:27.090 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:27.090 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:27.093 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:27.094 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:27.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:27.150 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:27.145+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:27.166 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-21T23:10:27.251 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:27.245+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:27.260 INFO:teuthology.orchestra.run.smithi150.stdout:1393 trunc from 0xb789aa9 to 0xd77a959 2024-09-21T23:10:27.268 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:27.268 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:27.287 INFO:teuthology.orchestra.run.smithi150.stdout:1394 trunc from 0xd77a959 to 0x9791aa 2024-09-21T23:10:27.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:27.362 INFO:teuthology.orchestra.run.smithi150.stdout:1396 read 0x5c813c thru 0x5d046c (0x8331 bytes) 2024-09-21T23:10:27.363 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:27.357+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:27.363 INFO:teuthology.orchestra.run.smithi150.stdout:1397 read 0x8ec80f thru 0x8f80d1 (0xb8c3 bytes) 2024-09-21T23:10:27.376 INFO:teuthology.orchestra.run.smithi150.stdout:1400 trunc from 0x9791aa to 0x9dc098e 2024-09-21T23:10:27.381 INFO:teuthology.orchestra.run.smithi150.stdout:1402 read 0x8c82772 thru 0x8c90247 (0xdad6 bytes) 2024-09-21T23:10:27.381 INFO:teuthology.orchestra.run.smithi150.stdout:1403 read 0x9bd4cba thru 0x9bdb24d (0x6594 bytes) 2024-09-21T23:10:27.385 INFO:teuthology.orchestra.run.smithi150.stdout:1404 trunc from 0x9dc098e to 0xcc84eb2 2024-09-21T23:10:27.390 INFO:teuthology.orchestra.run.smithi150.stdout:1405 write 0x5f124df thru 0x5f1b6bd (0x91df bytes) 2024-09-21T23:10:27.395 INFO:teuthology.orchestra.run.smithi150.stdout:1406 read 0x6ea6bf3 thru 0x6eb595f (0xed6d bytes) 2024-09-21T23:10:27.395 INFO:teuthology.orchestra.run.smithi150.stdout:1407 read 0x72fe018 thru 0x7306904 (0x88ed bytes) 2024-09-21T23:10:27.395 INFO:teuthology.orchestra.run.smithi150.stdout:1408 write 0xcbdd384 thru 0xcbed29f (0xff1c bytes) 2024-09-21T23:10:27.402 INFO:teuthology.orchestra.run.smithi150.stdout:1410 writesame 0x5328eb1 thru 0x532a1c2 data_size 0x1(0x1312 bytes) 2024-09-21T23:10:27.409 INFO:teuthology.orchestra.run.smithi150.stdout:1411 read 0x32967a4 thru 0x32983fe (0x1c5b bytes) 2024-09-21T23:10:27.409 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:27.409 INFO:teuthology.orchestra.run.smithi150.stdout:1413 write 0xc280abf thru 0xc282f5c (0x249e bytes) 2024-09-21T23:10:27.412 INFO:teuthology.orchestra.run.smithi150.stdout:1414 read 0x169932f thru 0x16a0213 (0x6ee5 bytes) 2024-09-21T23:10:27.412 INFO:teuthology.orchestra.run.smithi150.stdout:1415 read 0x7b4aed9 thru 0x7b4fd26 (0x4e4e bytes) 2024-09-21T23:10:27.428 INFO:teuthology.orchestra.run.smithi150.stdout:1416 trunc from 0xcc84eb2 to 0x1759072 2024-09-21T23:10:27.443 INFO:teuthology.orchestra.run.smithi150.stdout:1417 trunc from 0x1759072 to 0x28e4286 2024-09-21T23:10:27.452 INFO:teuthology.orchestra.run.smithi150.stdout:1418 punch from 0x111f528 to 0x112a977, (0xb44f bytes) 2024-09-21T23:10:27.452 INFO:teuthology.orchestra.run.smithi150.stdout:1419 writesame 0x24fa295 thru 0x25059ed data_size 0x11(0xb759 bytes) 2024-09-21T23:10:27.453 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:27.449+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:27.470 INFO:teuthology.orchestra.run.smithi150.stdout:1421 trunc from 0x28e4286 to 0x9459d46 2024-09-21T23:10:27.486 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:27.486 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:27.496 INFO:teuthology.orchestra.run.smithi150.stdout:1422 write 0xe8df7da thru 0xe8e5dc6 (0x65ed bytes) 2024-09-21T23:10:27.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:27.503 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:27.503 INFO:teuthology.orchestra.run.smithi150.stdout:1425 punch from 0x6c44e2b to 0x6c52add, (0xdcb2 bytes) 2024-09-21T23:10:27.503 INFO:teuthology.orchestra.run.smithi150.stdout:1426 write 0x4d59a3a thru 0x4d65b15 (0xc0dc bytes) 2024-09-21T23:10:27.509 INFO:teuthology.orchestra.run.smithi150.stdout:1427 write 0x89ac771 thru 0x89bae73 (0xe703 bytes) 2024-09-21T23:10:27.516 INFO:teuthology.orchestra.run.smithi150.stdout:1428 read 0x6a476dd thru 0x6a4ab79 (0x349d bytes) 2024-09-21T23:10:27.516 INFO:teuthology.orchestra.run.smithi150.stdout:1430 writesame 0x971e6fc thru 0x972bc23 data_size 0x13(0xd528 bytes) 2024-09-21T23:10:27.524 INFO:teuthology.orchestra.run.smithi150.stdout:1431 write 0x125a310 thru 0x125df3f (0x3c30 bytes) 2024-09-21T23:10:27.527 INFO:teuthology.orchestra.run.smithi150.stdout:1432 write 0x6382b10 thru 0x638881d (0x5d0e bytes) 2024-09-21T23:10:27.531 INFO:teuthology.orchestra.run.smithi150.stdout:1434 write 0x29a0b3e thru 0x29ac214 (0xb6d7 bytes) 2024-09-21T23:10:27.535 INFO:teuthology.orchestra.run.smithi150.stdout:1436 write 0x9d04186 thru 0x9d08c80 (0x4afb bytes) 2024-09-21T23:10:27.540 INFO:teuthology.orchestra.run.smithi150.stdout:1438 write 0x3b508ba thru 0x3b58291 (0x79d8 bytes) 2024-09-21T23:10:27.547 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:27.547 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:27.547 INFO:teuthology.orchestra.run.smithi150.stdout:1441 write 0x5916191 thru 0x591e98b (0x87fb bytes) 2024-09-21T23:10:27.551 INFO:teuthology.orchestra.run.smithi150.stdout:1443 write 0x6b43fba thru 0x6b4fc93 (0xbcda bytes) 2024-09-21T23:10:27.553 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:27.553+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:27.556 INFO:teuthology.orchestra.run.smithi150.stdout:1445 writesame 0x79722c3 thru 0x797a3f9 data_size 0x1(0x8137 bytes) 2024-09-21T23:10:27.570 INFO:teuthology.orchestra.run.smithi150.stdout:1446 punch from 0x9c0a33b to 0x9c11e63, (0x7b28 bytes) 2024-09-21T23:10:27.570 INFO:teuthology.orchestra.run.smithi150.stdout:1447 write 0x87ec0f2 thru 0x87f1962 (0x5871 bytes) 2024-09-21T23:10:27.571 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:27.571 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:27.571 INFO:teuthology.orchestra.run.smithi150.stdout:1451 write 0x13fd6e2 thru 0x1405321 (0x7c40 bytes) 2024-09-21T23:10:27.575 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xeb3dbc0 2024-09-21T23:10:27.575 INFO:teuthology.orchestra.run.smithi150.stdout:1452 trunc from 0xe8e5dc7 to 0xeb3dbc0 2024-09-21T23:10:27.581 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:27.581 INFO:teuthology.orchestra.run.smithi150.stdout:1455 read 0x5457ffb thru 0x545ac91 (0x2c97 bytes) 2024-09-21T23:10:27.581 INFO:teuthology.orchestra.run.smithi150.stdout:1456 read 0xb56ba7c thru 0xb573336 (0x78bb bytes) 2024-09-21T23:10:27.581 INFO:teuthology.orchestra.run.smithi150.stdout:1457 clone 17 order 21 su 262144 sc 13 2024-09-21T23:10:27.622 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:10:27.639 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:27.639 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:27.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:27.696 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:27.693+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:27.755 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:27.753+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:27.803 INFO:teuthology.orchestra.run.smithi150.stdout:flattening image image_client.0-clone16 2024-09-21T23:10:27.824 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:27.824 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:27.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:27.856 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:27.853+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:27.957 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:27.953+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:28.004 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:28.004 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:28.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:28.062 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:28.061+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:28.147 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:28.155 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:28.158 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:28.153+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:28.164 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:28.217 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-21T23:10:28.263 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:28.257+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:28.352 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:28.352 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:28.361 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:28.357+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:28.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:28.462 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:28.461+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:28.511 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:28.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:28.563 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:28.557+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:28.661 INFO:teuthology.orchestra.run.smithi150.stdout:1458 writesame 0x1f92fed thru 0x1fa1773 data_size 0x17(0xe787 bytes) 2024-09-21T23:10:28.664 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:28.661+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:28.697 INFO:teuthology.orchestra.run.smithi150.stdout:1459 trunc from 0xeb3dbc0 to 0x4a2abd 2024-09-21T23:10:28.704 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:28.704 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:28.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:28.770 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:28.769+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:28.820 INFO:teuthology.orchestra.run.smithi150.stdout:1460 writesame 0xa7095f0 thru 0xa718cfe data_size 0x1(0xf70f bytes) 2024-09-21T23:10:28.828 INFO:teuthology.orchestra.run.smithi150.stdout:1461 write 0xa82b604 thru 0xa83a43a (0xee37 bytes) 2024-09-21T23:10:28.833 INFO:teuthology.orchestra.run.smithi150.stdout:1462 read 0x8fbaa68 thru 0x8fc0cdf (0x6278 bytes) 2024-09-21T23:10:28.833 INFO:teuthology.orchestra.run.smithi150.stdout:1463 read 0x98511b6 thru 0x9857fe4 (0x6e2f bytes) 2024-09-21T23:10:28.833 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xed9e1a6 2024-09-21T23:10:28.839 INFO:teuthology.orchestra.run.smithi150.stdout:1464 trunc from 0xa83a43b to 0xed9e1a6 2024-09-21T23:10:28.849 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:28.849 INFO:teuthology.orchestra.run.smithi150.stdout:1467 read 0x88d0043 thru 0x88d47a9 (0x4767 bytes) 2024-09-21T23:10:28.849 INFO:teuthology.orchestra.run.smithi150.stdout:1469 write 0x45ab708 thru 0x45bb273 (0xfb6c bytes) 2024-09-21T23:10:28.855 INFO:teuthology.orchestra.run.smithi150.stdout:1470 read 0x47f30a0 thru 0x47f7c86 (0x4be7 bytes) 2024-09-21T23:10:28.855 INFO:teuthology.orchestra.run.smithi150.stdout:1471 writesame 0x3c05ce1 thru 0x3c0930d data_size 0x43(0x362d bytes) 2024-09-21T23:10:28.861 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:28.862 INFO:teuthology.orchestra.run.smithi150.stdout:1473 read 0x5dbb86c thru 0x5dc3931 (0x80c6 bytes) 2024-09-21T23:10:28.862 INFO:teuthology.orchestra.run.smithi150.stdout:1474 write 0xd055b78 thru 0xd057541 (0x19ca bytes) 2024-09-21T23:10:28.865 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:28.861+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:28.868 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:28.868 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:28.868 INFO:teuthology.orchestra.run.smithi150.stdout:1477 write 0x93fde6c thru 0x940d446 (0xf5db bytes) 2024-09-21T23:10:28.874 INFO:teuthology.orchestra.run.smithi150.stdout:1478 write 0x41bc47e thru 0x41bc9ed (0x570 bytes) 2024-09-21T23:10:28.879 INFO:teuthology.orchestra.run.smithi150.stdout:1479 punch from 0x20ca77c to 0x20d782b, (0xd0af bytes) 2024-09-21T23:10:28.879 INFO:teuthology.orchestra.run.smithi150.stdout:1480 write 0xba6e222 thru 0xba729b5 (0x4794 bytes) 2024-09-21T23:10:28.885 INFO:teuthology.orchestra.run.smithi150.stdout:1481 writesame 0x59418c6 thru 0x5949e38 data_size 0x1(0x8573 bytes) 2024-09-21T23:10:28.892 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:28.892 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:28.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:28.924 INFO:teuthology.orchestra.run.smithi150.stdout:1483 read 0x7bbc22a thru 0x7bc9d70 (0xdb47 bytes) 2024-09-21T23:10:28.934 INFO:teuthology.orchestra.run.smithi150.stdout:1484 trunc from 0xed9e1a6 to 0xb54cc36 2024-09-21T23:10:28.952 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:28.952 INFO:teuthology.orchestra.run.smithi150.stdout:1486 write 0x707e98f thru 0x7087104 (0x8776 bytes) 2024-09-21T23:10:28.960 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:28.960 INFO:teuthology.orchestra.run.smithi150.stdout:1490 writesame 0x98e773f thru 0x98f2bd8 data_size 0x1(0xb49a bytes) 2024-09-21T23:10:28.985 INFO:teuthology.orchestra.run.smithi150.stdout:1492 read 0x95b252 thru 0x962233 (0x6fe2 bytes) 2024-09-21T23:10:28.986 INFO:teuthology.orchestra.run.smithi150.stdout:1493 read 0xa5c8568 thru 0xa5cf3cd (0x6e66 bytes) 2024-09-21T23:10:28.986 INFO:teuthology.orchestra.run.smithi150.stdout:1497 writesame 0x69d97a2 thru 0x69e6111 data_size 0x58(0xc970 bytes) 2024-09-21T23:10:28.990 INFO:teuthology.orchestra.run.smithi150.stdout:1498 punch from 0x3a6d06b to 0x3a74eed, (0x7e82 bytes) 2024-09-21T23:10:28.990 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:28.990 INFO:teuthology.orchestra.run.smithi150.stdout:1500 writesame 0x7f97409 thru 0x7f999aa data_size 0x1(0x25a2 bytes) 2024-09-21T23:10:28.997 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:29.002 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:29.001+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:29.014 INFO:teuthology.orchestra.run.smithi150.stdout:1502 write 0xcfbe5a1 thru 0xcfcdb86 (0xf5e6 bytes) 2024-09-21T23:10:29.022 INFO:teuthology.orchestra.run.smithi150.stdout:1504 punch from 0x5b2395 to 0x5b4d8a, (0x29f5 bytes) 2024-09-21T23:10:29.022 INFO:teuthology.orchestra.run.smithi150.stdout:1505 read 0x9d60177 thru 0x9d60b92 (0xa1c bytes) 2024-09-21T23:10:29.034 INFO:teuthology.orchestra.run.smithi150.stdout:1506 write 0xd74ce54 thru 0xd75aa11 (0xdbbe bytes) 2024-09-21T23:10:29.040 INFO:teuthology.orchestra.run.smithi150.stdout:1507 write 0x448b5ee thru 0x448e3a3 (0x2db6 bytes) 2024-09-21T23:10:29.068 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:29.061+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:29.068 INFO:teuthology.orchestra.run.smithi150.stdout:1509 trunc from 0xd75aa12 to 0x55a98dc 2024-09-21T23:10:29.082 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:29.082 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:29.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:29.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:29.116 INFO:teuthology.orchestra.run.smithi150.stdout:1513 writesame 0x965c2eb thru 0x9666235 data_size 0x17(0x9f4b bytes) 2024-09-21T23:10:29.117 INFO:teuthology.orchestra.run.smithi150.stdout:1514 writesame 0x1cdc327 thru 0x1ce33d6 data_size 0x18(0x70b0 bytes) 2024-09-21T23:10:29.131 INFO:teuthology.orchestra.run.smithi150.stdout:1515 write 0x9917ad5 thru 0x992511a (0xd646 bytes) 2024-09-21T23:10:29.136 INFO:teuthology.orchestra.run.smithi150.stdout:1516 write 0x9302cba thru 0x93031ea (0x531 bytes) 2024-09-21T23:10:29.139 INFO:teuthology.orchestra.run.smithi150.stdout:1519 punch from 0x40b3bb9 to 0x40baf42, (0x7389 bytes) 2024-09-21T23:10:29.143 INFO:teuthology.orchestra.run.smithi150.stdout:1520 trunc from 0x992511b to 0xb5b9b40 2024-09-21T23:10:29.152 INFO:teuthology.orchestra.run.smithi150.stdout:1521 write 0x687c1b7 thru 0x68803dc (0x4226 bytes) 2024-09-21T23:10:29.159 INFO:teuthology.orchestra.run.smithi150.stdout:1522 flatten 2024-09-21T23:10:29.168 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:29.165+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:29.171 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:29.171 INFO:teuthology.orchestra.run.smithi150.stdout:1524 punch from 0x2374af0 to 0x23838d0, (0xede0 bytes) 2024-09-21T23:10:29.176 INFO:teuthology.orchestra.run.smithi150.stdout:1525 trunc from 0xb5b9b40 to 0xeb30da4 2024-09-21T23:10:29.187 INFO:teuthology.orchestra.run.smithi150.stdout:1526 punch from 0x8c85037 to 0x8c88fd6, (0x3f9f bytes) 2024-09-21T23:10:29.187 INFO:teuthology.orchestra.run.smithi150.stdout:1527 read 0xb9740cc thru 0xb9761fc (0x2131 bytes) 2024-09-21T23:10:29.187 INFO:teuthology.orchestra.run.smithi150.stdout:1529 read 0xbf49f7d thru 0xbf58fd0 (0xf054 bytes) 2024-09-21T23:10:29.187 INFO:teuthology.orchestra.run.smithi150.stdout:1530 punch from 0x8acb857 to 0x8ad1453, (0x5bfc bytes) 2024-09-21T23:10:29.187 INFO:teuthology.orchestra.run.smithi150.stdout:1533 read 0x560d5c3 thru 0x561263b (0x5079 bytes) 2024-09-21T23:10:29.188 INFO:teuthology.orchestra.run.smithi150.stdout:1534 punch from 0xca71fec to 0xca794dd, (0x74f1 bytes) 2024-09-21T23:10:29.188 INFO:teuthology.orchestra.run.smithi150.stdout:1538 write 0xa5ced44 thru 0xa5d0baa (0x1e67 bytes) 2024-09-21T23:10:29.194 INFO:teuthology.orchestra.run.smithi150.stdout:1539 write 0x24e39bc thru 0x24f33b9 (0xf9fe bytes) 2024-09-21T23:10:29.199 INFO:teuthology.orchestra.run.smithi150.stdout:1540 read 0xe5ec96d thru 0xe5f8376 (0xba0a bytes) 2024-09-21T23:10:29.215 INFO:teuthology.orchestra.run.smithi150.stdout:1542 trunc from 0xeb30da4 to 0x203fa57 2024-09-21T23:10:29.239 INFO:teuthology.orchestra.run.smithi150.stdout:1543 read 0x17057ff thru 0x170f62a (0x9e2c bytes) 2024-09-21T23:10:29.240 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:29.251 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:29.251 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:29.254 INFO:teuthology.orchestra.run.smithi150.stdout:1545 trunc from 0x203fa57 to 0xd812c68 2024-09-21T23:10:29.255 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:29.256 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:29.264 INFO:teuthology.orchestra.run.smithi150.stdout:1547 writesame 0x2fa695f thru 0x2fae8ec data_size 0x2(0x7f8e bytes) 2024-09-21T23:10:29.267 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:29.275 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:29.275 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:29.276 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:29.273+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:29.276 INFO:teuthology.orchestra.run.smithi150.stdout:1548 punch from 0xb4a9783 to 0xb4aa1c5, (0xa42 bytes) 2024-09-21T23:10:29.276 INFO:teuthology.orchestra.run.smithi150.stdout:1550 write 0xd4352bd thru 0xd4439cd (0xe711 bytes) 2024-09-21T23:10:29.283 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:29.294 INFO:teuthology.orchestra.run.smithi150.stdout:1552 trunc from 0xd812c68 to 0x568d0b5 2024-09-21T23:10:29.294 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:29.295 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:29.311 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:29.311 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:29.312 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:29.313 INFO:teuthology.orchestra.run.smithi150.stdout:1554 write 0x1145663 thru 0x11498e2 (0x4280 bytes) 2024-09-21T23:10:29.316 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:29.316 INFO:teuthology.orchestra.run.smithi150.stdout:1557 punch from 0x2c9f3e3 to 0x2caaa89, (0xb6a6 bytes) 2024-09-21T23:10:29.318 INFO:teuthology.orchestra.run.smithi150.stdout:1558 trunc from 0x568d0b5 to 0x7294322 2024-09-21T23:10:29.330 INFO:teuthology.orchestra.run.smithi150.stdout:1561 write 0x21d998b thru 0x21e8a23 (0xf099 bytes) 2024-09-21T23:10:29.335 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:29.335 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:29.336 INFO:teuthology.orchestra.run.smithi150.stdout:1563 read 0x5ea40e5 thru 0x5ea667c (0x2598 bytes) 2024-09-21T23:10:29.336 INFO:teuthology.orchestra.run.smithi150.stdout:1565 punch from 0x2fe0579 to 0x2fe18db, (0x1362 bytes) 2024-09-21T23:10:29.336 INFO:teuthology.orchestra.run.smithi150.stdout:1566 punch from 0x195ed4a to 0x196db87, (0xee3d bytes) 2024-09-21T23:10:29.343 INFO:teuthology.orchestra.run.smithi150.stdout:1567 trunc from 0x7294322 to 0xc96e5cc 2024-09-21T23:10:29.355 INFO:teuthology.orchestra.run.smithi150.stdout:1568 read 0xa5d0ff4 thru 0xa5d9009 (0x8016 bytes) 2024-09-21T23:10:29.356 INFO:teuthology.orchestra.run.smithi150.stdout:1570 punch from 0x889058d to 0x889cf94, (0xca07 bytes) 2024-09-21T23:10:29.369 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:29.365+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:29.370 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:29.370 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:29.370 INFO:teuthology.orchestra.run.smithi150.stdout:1571 trunc from 0xc96e5cc to 0x21a95a 2024-09-21T23:10:29.395 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:29.395 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:29.399 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:29.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:29.407 INFO:teuthology.orchestra.run.smithi150.stdout:1574 punch from 0x20c9ee to 0x210883, (0x3e95 bytes) 2024-09-21T23:10:29.407 INFO:teuthology.orchestra.run.smithi150.stdout:1575 punch from 0x12ceb8 to 0x136dde, (0x9f26 bytes) 2024-09-21T23:10:29.407 INFO:teuthology.orchestra.run.smithi150.stdout:1577 punch from 0x1e3f5d to 0x1eb950, (0x79f3 bytes) 2024-09-21T23:10:29.407 INFO:teuthology.orchestra.run.smithi150.stdout:1578 read 0xf443f thru 0xf8059 (0x3c1b bytes) 2024-09-21T23:10:29.408 INFO:teuthology.orchestra.run.smithi150.stdout:1581 clone 18 order 22 su 1048576 sc 13 2024-09-21T23:10:29.409 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:29.409 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:29.410 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:29.410 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:29.410 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:29.422 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:29.508 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:29.505+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:29.572 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:29.569+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:29.584 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12737 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2043904~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484976262, 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035132690000000001, 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12737 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097393+0000", 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097394+0000", 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:10:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097397+0000", 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9890000000000001e-06 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097400+0000", 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132379+0000", 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034979237000000003 2024-09-21T23:10:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132438+0000", 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8183999999999997e-05 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132526+0000", 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8171999999999995e-05 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12738 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2058240~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484965975000001, 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035265405999999999, 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12738 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097403+0000", 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097404+0000", 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097405+0000", 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097408+0000", 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.971e-06 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097410+0000", 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132558+0000", 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035147993000000002 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132611+0000", 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2293000000000002e-05 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132669+0000", 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8139e-05 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12739 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2072576~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484955909, 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035408621000000001, 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12739 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097413+0000", 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097414+0000", 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097415+0000", 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.046e-06 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097418+0000", 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9629999999999998e-06 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097424+0000", 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.891e-06 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132706+0000", 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035282154000000003 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132762+0000", 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6475999999999999e-05 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132822+0000", 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9877999999999998e-05 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12740 3.1b 3:d830ef5f:::rbd_data.1149e13ad964.0000000000000009:b [read 2086912~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484939845, 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035536514999999998, 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12740 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097430+0000", 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6399999999999998e-07 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097432+0000", 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.889e-06 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097438+0000", 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9270000000000002e-06 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.097442+0000", 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4340000000000002e-06 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132856+0000", 2024-09-21T23:10:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035413922 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132906+0000", 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9997999999999998e-05 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132966+0000", 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9981000000000001e-05 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12797 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1313792~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484356254000001, 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034983512000000001, 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12797 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098013+0000", 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098015+0000", 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1850000000000001e-06 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098018+0000", 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1540000000000002e-06 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098020+0000", 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6390000000000001e-06 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132848+0000", 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034828143999999998 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132909+0000", 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0825999999999997e-05 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.132997+0000", 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7348999999999993e-05 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12798 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1328128~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484345655999999, 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035158699000000002, 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12798 2024-09-21T23:10:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098024+0000", 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098025+0000", 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0440000000000001e-06 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098028+0000", 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098031+0000", 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7499999999999999e-06 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133037+0000", 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035005938 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133095+0000", 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8208000000000002e-05 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133182+0000", 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7559000000000007e-05 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12799 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1342464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484335021, 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035342140000000001, 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12799 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098034+0000", 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098035+0000", 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098036+0000", 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.172e-06 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098040+0000", 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8319999999999999e-06 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098042+0000", 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8509999999999999e-06 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133220+0000", 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035177096999999997 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133288+0000", 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8534000000000005e-05 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133376+0000", 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8441000000000002e-05 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12800 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1356800~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484323905, 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03547894, 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12800 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098045+0000", 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098046+0000", 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.16e-07 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098047+0000", 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098051+0000", 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.687e-06 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098053+0000", 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.689e-06 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133401+0000", 2024-09-21T23:10:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035347575999999999 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133446+0000", 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4962000000000001e-05 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133524+0000", 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8568999999999999e-05 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12801 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1371136~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484312644999999, 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035624792000000002, 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12801 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098057+0000", 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098058+0000", 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.105e-06 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098062+0000", 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.844e-06 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098065+0000", 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7310000000000002e-06 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133557+0000", 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035492255 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133605+0000", 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8569000000000002e-05 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133682+0000", 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6070999999999997e-05 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12802 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1385472~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484301516999999, 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035748412, 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12802 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098068+0000", 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098069+0000", 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098072+0000", 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.033e-06 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098075+0000", 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8490000000000002e-06 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133709+0000", 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035633962999999998 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133753+0000", 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3809e-05 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133816+0000", 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3301999999999998e-05 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12803 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1399808~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484290419000001, 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035862430000000001, 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12803 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098079+0000", 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:29.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098080+0000", 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098083+0000", 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0479999999999999e-06 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098087+0000", 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2030000000000001e-06 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133842+0000", 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035755012000000003 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133885+0000", 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000002e-05 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133941+0000", 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6648999999999999e-05 2024-09-21T23:10:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12804 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1414144~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484279699999998, 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035982214999999998, 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12804 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098090+0000", 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098091+0000", 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2449999999999999e-06 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098094+0000", 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0120000000000001e-06 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098097+0000", 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7199999999999998e-06 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.133968+0000", 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035871003999999998 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134016+0000", 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8368999999999997e-05 2024-09-21T23:10:29.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134072+0000", 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5649999999999997e-05 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12805 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1428480~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484269502, 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036102355000000003, 2024-09-21T23:10:29.635 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12805 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098100+0000", 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098101+0000", 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098110+0000", 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1290000000000002e-06 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098113+0000", 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8310000000000001e-06 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134100+0000", 2024-09-21T23:10:29.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035986481000000001 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134144+0000", 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3890000000000002e-05 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134202+0000", 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8613000000000003e-05 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12806 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1442816~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484253179, 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036273185999999999, 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12806 2024-09-21T23:10:29.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098116+0000", 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098117+0000", 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098118+0000", 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.124e-06 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098121+0000", 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:10:29.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098124+0000", 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8320000000000002e-06 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134229+0000", 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036105406 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134295+0000", 2024-09-21T23:10:29.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6282e-05 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.134389+0000", 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4168999999999999e-05 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12807 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1457152~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484240888999999, 2024-09-21T23:10:29.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037538491, 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12807 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098128+0000", 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098129+0000", 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2100000000000001e-07 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098130+0000", 2024-09-21T23:10:29.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1149999999999999e-06 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098133+0000", 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.258e-06 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098136+0000", 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7810000000000001e-06 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135467+0000", 2024-09-21T23:10:29.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037331371000000002 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135549+0000", 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2211000000000003e-05 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135667+0000", 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117534 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12808 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1471488~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484230362000002, 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037746517, 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12808 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098139+0000", 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098142+0000", 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098145+0000", 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098148+0000", 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7750000000000001e-06 2024-09-21T23:10:29.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135714+0000", 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037566612999999999 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135788+0000", 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4000999999999998e-05 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135886+0000", 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7272e-05 2024-09-21T23:10:29.652 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12809 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1485824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484218608999999, 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037999241000000003, 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12809 2024-09-21T23:10:29.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098151+0000", 2024-09-21T23:10:29.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1e-07 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098152+0000", 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0130000000000001e-06 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098155+0000", 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0290000000000001e-06 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098158+0000", 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.565e-06 2024-09-21T23:10:29.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.135936+0000", 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037778467000000003 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136047+0000", 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011086099999999999 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136150+0000", 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103096 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.656 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12810 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1500160~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484208848000002, 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038237449, 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12810 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.657 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098161+0000", 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098162+0000", 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.189e-06 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098165+0000", 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9780000000000001e-06 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098168+0000", 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:10:29.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136187+0000", 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038019325999999999 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136272+0000", 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4497000000000001e-05 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136398+0000", 2024-09-21T23:10:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012611600000000001 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:12811 3.3 3:c58b9510:::rbd_data.1149e13ad964.0000000000000008:b [read 1514496~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.484198380999999, 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038486382, 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 12811 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098171+0000", 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3999999999999998e-07 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098172+0000", 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.24e-06 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098175+0000", 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.976e-06 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.098178+0000", 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8200000000000001e-06 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136448+0000", 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038269965000000003 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136543+0000", 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5069999999999996e-05 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:13.136657+0000", 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114071 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21978 3.1f 3:fa161337:::rbd_data.1149c115bf92.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.849066153999999, 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052821716999999997, 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:29.665 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21978 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:29.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733303+0000", 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733307+0000", 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733321+0000", 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4049999999999999e-05 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733357+0000", 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6363999999999999e-05 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733375+0000", 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733493+0000", 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117642 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.733629+0000", 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000136596 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.742696+0000", 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0090670260000000006 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.785924+0000", 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043228124 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.785957+0000", 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2252999999999998e-05 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.786048+0000", 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1731999999999995e-05 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.786125+0000", 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6606999999999993e-05 2024-09-21T23:10:29.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.671 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:29.671 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:29.671 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:29.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:10:29.672 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:29.673 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:29.669+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:29.773 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:29.769+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:29.813 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-21T23:10:29.857 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:29.857 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:29.857 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:29.857 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:29.873 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:29.869+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:29.875 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:29.886 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T23:10:29.886 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:10:29.886 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-21T23:10:29.886 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:10:29.886 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:10:29.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-21T23:10:29.974 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:29.973+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:30.018 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:30.018 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:10:30.018 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:30.053 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:30.053 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:30.053 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:30.053 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:30.053 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:30.069 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:30.116 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:30.113+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:30.177 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:30.173+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4287 3.0 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1301057~9663] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.025674+0000", 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.163179547999999, 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019368100999999999, 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4287 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.025674+0000", 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.025674+0000", 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.025676+0000", 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027586+0000", 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019099589999999999 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027620+0000", 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3729000000000002e-05 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027635+0000", 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5024999999999999e-05 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027749+0000", 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114707 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.027934+0000", 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00018491300000000001 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.028951+0000", 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00101639 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044295+0000", 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01534408 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044933+0000", 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00063873300000000005 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044992+0000", 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8445e-05 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.045042+0000", 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130999999999998e-05 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.196 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4886 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2949120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.181352+0000", 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.007501227999999, 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01390837, 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4886 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181352+0000", 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181352+0000", 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181353+0000", 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7199999999999998e-07 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181357+0000", 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.676e-06 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181371+0000", 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4139e-05 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181380+0000", 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2520000000000005e-06 2024-09-21T23:10:30.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.194935+0000", 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013555322999999999 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195123+0000", 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187982 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195261+0000", 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000137226 2024-09-21T23:10:30.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4887 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3407872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.007461905, 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014700754, 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.201 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4887 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181392+0000", 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181396+0000", 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.67e-06 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181407+0000", 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181418+0000", 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1088e-05 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195304+0000", 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013885817999999999 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.195756+0000", 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00045195899999999999 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196092+0000", 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000336201 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4888 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 3866624~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.007422769000001, 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015118104, 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.205 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4888 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181431+0000", 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181442+0000", 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.024e-05 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181454+0000", 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2788e-05 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.181464+0000", 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.876e-06 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196122+0000", 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014657701 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196192+0000", 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9920999999999997e-05 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.196549+0000", 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035691199999999997 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5553 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.990260655, 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013603697999999999, 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5553 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198593+0000", 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7700000000000004e-07 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198596+0000", 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5270000000000001e-06 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198605+0000", 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2089999999999994e-06 2024-09-21T23:10:30.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198612+0000", 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8079999999999999e-06 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.211710+0000", 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013098205 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212064+0000", 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035353899999999999 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212197+0000", 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013283300000000001 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5554 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198620+0000", 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.990232998, 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0138052, 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5554 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198620+0000", 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198620+0000", 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198621+0000", 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8100000000000003e-07 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198625+0000", 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.602e-06 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198633+0000", 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8210000000000008e-06 2024-09-21T23:10:30.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198640+0000", 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7009999999999999e-06 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212306+0000", 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013665773000000001 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212365+0000", 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8752000000000003e-05 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212426+0000", 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0970000000000001e-05 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5555 3.19 3:9cba6ead:::rbd_data.114912f5d0f.0000000000000000:7 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.990204594000001, 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014241567, 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5555 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:30.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198649+0000", 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2099999999999997e-07 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198652+0000", 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6620000000000001e-06 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198660+0000", 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2449999999999998e-06 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198667+0000", 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.737e-06 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212444+0000", 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013776563 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212576+0000", 2024-09-21T23:10:30.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013281 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.212890+0000", 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031402900000000001 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20822 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.496306970999999, 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015204284, 2024-09-21T23:10:30.221 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20822 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692547+0000", 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8900000000000001e-07 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692549+0000", 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8309999999999999e-06 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692552+0000", 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7100000000000001e-06 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692555+0000", 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.084e-06 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.703174+0000", 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010618588 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.703295+0000", 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120871 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.703583+0000", 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028789400000000002 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707646+0000", 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040630279999999998 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:30.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707674+0000", 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8320999999999999e-05 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707707+0000", 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3197000000000001e-05 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707751+0000", 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3470999999999998e-05 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20823 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.49629436, 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015300924, 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20823 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692559+0000", 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692561+0000", 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8509999999999999e-06 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692565+0000", 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6600000000000001e-06 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692569+0000", 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.704019+0000", 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011450435 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.704079+0000", 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9221000000000001e-05 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.704345+0000", 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026609600000000001 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707764+0000", 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034193470000000001 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707792+0000", 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8053000000000001e-05 2024-09-21T23:10:30.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707822+0000", 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0219999999999999e-05 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707860+0000", 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7808999999999997e-05 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20824 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692581+0000", 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.496272245, 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015439424, 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20824 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692581+0000", 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692581+0000", 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692582+0000", 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0699999999999998e-07 2024-09-21T23:10:30.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692590+0000", 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4500000000000004e-06 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692595+0000", 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2170000000000002e-06 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692599+0000", 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.788e-06 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.705269+0000", 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012669654000000001 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.705424+0000", 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000155067 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.705963+0000", 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00053946500000000004 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707767+0000", 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00180382 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707927+0000", 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016015800000000001 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707989+0000", 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1366999999999997e-05 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708021+0000", 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2030999999999999e-05 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20825 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.238 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692604+0000", 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.496249111000001, 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016684879999999999, 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20825 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692604+0000", 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692604+0000", 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692605+0000", 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8299999999999998e-07 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692607+0000", 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1629999999999999e-06 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692611+0000", 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9249999999999997e-06 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692614+0000", 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.455e-06 2024-09-21T23:10:30.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.706708+0000", 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014093879 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.706822+0000", 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113947 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.707231+0000", 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040900499999999999 2024-09-21T23:10:30.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709009+0000", 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017782900000000001 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709061+0000", 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1283999999999999e-05 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709150+0000", 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9431999999999993e-05 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709289+0000", 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013921699999999999 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:20826 3.19 3:9ea64d7e:::rbd_object_map.1149f84cbfe6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:30.244 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.496235531, 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017943988000000001, 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20826 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692618+0000", 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4699999999999998e-07 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692620+0000", 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9360000000000002e-06 2024-09-21T23:10:30.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692633+0000", 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3106e-05 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.692638+0000", 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7700000000000001e-06 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708072+0000", 2024-09-21T23:10:30.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015434045 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708175+0000", 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102643 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.708530+0000", 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355688 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.709445+0000", 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000914655 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.710359+0000", 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00091388999999999999 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.710477+0000", 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000118424 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:15.710562+0000", 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4584000000000002e-05 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21677 3.1a 3:5a717256:::rbd_data.1149f84cbfe6.0000000000000010:c [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.331989+0000", 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.856864613999999, 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013983705000000001, 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21677 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331989+0000", 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331989+0000", 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331990+0000", 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.35e-06 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.331998+0000", 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5669999999999997e-06 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332019+0000", 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1613000000000001e-05 2024-09-21T23:10:30.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332034+0000", 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4276e-05 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.345477+0000", 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01344365 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.345697+0000", 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022008800000000001 2024-09-21T23:10:30.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.345973+0000", 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000275161 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21678 3.1a 3:5a717256:::rbd_data.1149f84cbfe6.0000000000000010:c [read 187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.332056+0000", 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.856797575, 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014483438, 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.254 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21678 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332056+0000", 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332056+0000", 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332058+0000", 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6029999999999999e-06 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332064+0000", 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7070000000000003e-06 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332082+0000", 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8117e-05 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332097+0000", 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4759999999999999e-05 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346050+0000", 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013953402 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346266+0000", 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215038 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346539+0000", 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273811 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21679 3.1a 3:5a717256:::rbd_data.1149f84cbfe6.0000000000000010:c [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.332116+0000", 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.856737197999999, 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014976817, 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21679 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332116+0000", 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332116+0000", 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332117+0000", 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8299999999999995e-07 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332123+0000", 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332140+0000", 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7563000000000001e-05 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332154+0000", 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3692999999999999e-05 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346617+0000", 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014463055000000001 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.346826+0000", 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000208975 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347093+0000", 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026689800000000001 2024-09-21T23:10:30.262 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21705 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 18432~18432] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.332944+0000", 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.855909169, 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014791653, 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21705 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332944+0000", 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332944+0000", 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332947+0000", 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2900000000000001e-06 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332958+0000", 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.169e-05 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.332989+0000", 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0904000000000001e-05 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333002+0000", 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.309e-05 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347166+0000", 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014164177 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347409+0000", 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024228499999999999 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347736+0000", 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032721699999999998 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21706 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333046+0000", 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.855807662, 2024-09-21T23:10:30.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015328334000000001, 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21706 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333046+0000", 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333046+0000", 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333047+0000", 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-06 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333054+0000", 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333075+0000", 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0636e-05 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333098+0000", 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2937000000000001e-05 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.347820+0000", 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014721916 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348035+0000", 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021504200000000001 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348374+0000", 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033955999999999998 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.271 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21707 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333138+0000", 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.855715440000001, 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015804684999999999, 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21707 2024-09-21T23:10:30.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333138+0000", 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333138+0000", 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333140+0000", 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4640000000000001e-06 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333145+0000", 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8760000000000002e-06 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333172+0000", 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.654e-05 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333186+0000", 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.413e-05 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348451+0000", 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015265435000000001 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348670+0000", 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000218805 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.348943+0000", 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027243500000000002 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21708 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333206+0000", 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.855647454, 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016345584, 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.276 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21708 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333206+0000", 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333206+0000", 2024-09-21T23:10:30.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333207+0000", 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333213+0000", 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8170000000000004e-06 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333233+0000", 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9729999999999999e-05 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333246+0000", 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3603e-05 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349021+0000", 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015774921000000001 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349272+0000", 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025024500000000001 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349552+0000", 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000280098 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21709 3.1a 3:591888d6:::rbd_data.1149f84cbfe6.0000000000000016:c [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.333265+0000", 2024-09-21T23:10:30.280 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.855588291, 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016949235999999999, 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21709 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333265+0000", 2024-09-21T23:10:30.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333265+0000", 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333266+0000", 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.15e-06 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333272+0000", 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5740000000000003e-06 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333291+0000", 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9409e-05 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.333304+0000", 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2984e-05 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349628+0000", 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016324113000000001 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.349920+0000", 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029203500000000001 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.350214+0000", 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029397100000000002 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.284 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.285 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.285 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:30.286 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:30.287 INFO:teuthology.orchestra.run.smithi150.stdout:1582 write 0x12437ef thru 0x124a1c3 (0x69d5 bytes) 2024-09-21T23:10:30.287 INFO:teuthology.orchestra.run.smithi150.stdout:1584 trunc from 0x124a1c4 to 0x226f58f 2024-09-21T23:10:30.287 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:30.285+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:30.288 INFO:teuthology.orchestra.run.smithi150.stdout:1588 write 0x27834f1 thru 0x2792046 (0xeb56 bytes) 2024-09-21T23:10:30.312 INFO:teuthology.orchestra.run.smithi150.stdout:1589 write 0x38c3332 thru 0x38d1aa9 (0xe778 bytes) 2024-09-21T23:10:30.319 INFO:teuthology.orchestra.run.smithi150.stdout:1590 punch from 0x32189b1 to 0x321de97, (0x54e6 bytes) 2024-09-21T23:10:30.319 INFO:teuthology.orchestra.run.smithi150.stdout:1591 writesame 0x2d7ac71 thru 0x2d810fc data_size 0x2(0x648c bytes) 2024-09-21T23:10:30.341 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:30.342 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:30.342 INFO:teuthology.orchestra.run.smithi150.stdout:1597 punch from 0x10b70ee to 0x10b8dd6, (0x1ce8 bytes) 2024-09-21T23:10:30.344 INFO:teuthology.orchestra.run.smithi150.stdout:1598 trunc from 0x38d1aaa to 0x22e8341 2024-09-21T23:10:30.387 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:30.381+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:30.406 INFO:teuthology.orchestra.run.smithi150.stdout:1599 write 0xd8cd19e thru 0xd8cd295 (0xf8 bytes) 2024-09-21T23:10:30.412 INFO:teuthology.orchestra.run.smithi150.stdout:1600 write 0x2cfb505 thru 0x2d04022 (0x8b1e bytes) 2024-09-21T23:10:30.416 INFO:teuthology.orchestra.run.smithi150.stdout:1603 writesame 0x7395de6 thru 0x73a09ff data_size 0x15(0xac1a bytes) 2024-09-21T23:10:30.421 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:30.421 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:30.424 INFO:teuthology.orchestra.run.smithi150.stdout:1604 read 0xa879a53 thru 0xa882ec5 (0x9473 bytes) 2024-09-21T23:10:30.424 INFO:teuthology.orchestra.run.smithi150.stdout:1605 read 0xab86934 thru 0xab93b15 (0xd1e2 bytes) 2024-09-21T23:10:30.424 INFO:teuthology.orchestra.run.smithi150.stdout:1607 writesame 0x59c97f2 thru 0x59d1d4c data_size 0x1(0x855b bytes) 2024-09-21T23:10:30.436 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:30.462 INFO:teuthology.orchestra.run.smithi150.stdout:1608 write 0x75ad43 thru 0x760477 (0x5735 bytes) 2024-09-21T23:10:30.469 INFO:teuthology.orchestra.run.smithi150.stdout:1609 writesame 0x7e2cf51 thru 0x7e2d8bf data_size 0x17(0x96f bytes) 2024-09-21T23:10:30.475 INFO:teuthology.orchestra.run.smithi150.stdout:1610 punch from 0x14952a8 to 0x149772e, (0x2486 bytes) 2024-09-21T23:10:30.475 INFO:teuthology.orchestra.run.smithi150.stdout:1611 writesame 0xae6b450 thru 0xae71221 data_size 0x3(0x5dd2 bytes) 2024-09-21T23:10:30.487 INFO:teuthology.orchestra.run.smithi150.stdout:1613 write 0xc071870 thru 0xc07da21 (0xc1b2 bytes) 2024-09-21T23:10:30.488 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:30.485+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:30.493 INFO:teuthology.orchestra.run.smithi150.stdout:1615 writesame 0xf2c8e3 thru 0xf392f0 data_size 0x6f(0xca0e bytes) 2024-09-21T23:10:30.496 INFO:teuthology.orchestra.run.smithi150.stdout:1618 writesame 0xa89d6ca thru 0xa8aa2df data_size 0x2(0xcc16 bytes) 2024-09-21T23:10:30.520 INFO:teuthology.orchestra.run.smithi150.stdout:1620 read 0xa31577a thru 0xa31a5c5 (0x4e4c bytes) 2024-09-21T23:10:30.520 INFO:teuthology.orchestra.run.smithi150.stdout:1621 write 0x58d2b4c thru 0x58dae5e (0x8313 bytes) 2024-09-21T23:10:30.525 INFO:teuthology.orchestra.run.smithi150.stdout:1622 writesame 0x3776bf5 thru 0x3780d3e data_size 0x5(0xa14a bytes) 2024-09-21T23:10:30.539 INFO:teuthology.orchestra.run.smithi150.stdout:1625 write 0xbb4e9cb thru 0xbb5043c (0x1a72 bytes) 2024-09-21T23:10:30.542 INFO:teuthology.orchestra.run.smithi150.stdout:1627 read 0x4b0f3d6 thru 0x4b1c6a8 (0xd2d3 bytes) 2024-09-21T23:10:30.543 INFO:teuthology.orchestra.run.smithi150.stdout:1629 write 0x748880a thru 0x748c5a1 (0x3d98 bytes) 2024-09-21T23:10:30.547 INFO:teuthology.orchestra.run.smithi150.stdout:1632 writesame 0x65bb33 thru 0x65dbd0 data_size 0x19(0x209e bytes) 2024-09-21T23:10:30.551 INFO:teuthology.orchestra.run.smithi150.stdout:1633 write 0xd0db064 thru 0xd0e94da (0xe477 bytes) 2024-09-21T23:10:30.568 INFO:teuthology.orchestra.run.smithi150.stdout:1634 trunc from 0xd8cd296 to 0x7f6b34b 2024-09-21T23:10:30.589 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:30.585+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:30.589 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:30.590 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:30.595 INFO:teuthology.orchestra.run.smithi150.stdout:1635 write 0x38dc08a thru 0x38e1a41 (0x59b8 bytes) 2024-09-21T23:10:30.605 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:30.615 INFO:teuthology.orchestra.run.smithi150.stdout:1637 writesame 0x8593a6e thru 0x8596995 data_size 0x3(0x2f28 bytes) 2024-09-21T23:10:30.615 INFO:teuthology.orchestra.run.smithi150.stdout:1638 write 0x4f08a97 thru 0x4f12578 (0x9ae2 bytes) 2024-09-21T23:10:30.626 INFO:teuthology.orchestra.run.smithi150.stdout:1639 trunc from 0x8596996 to 0xb4f5230 2024-09-21T23:10:30.636 INFO:teuthology.orchestra.run.smithi150.stdout:1640 write 0x330b6c3 thru 0x3316de6 (0xb724 bytes) 2024-09-21T23:10:30.641 INFO:teuthology.orchestra.run.smithi150.stdout:1641 punch from 0x5a83a62 to 0x5a8f22f, (0xb7cd bytes) 2024-09-21T23:10:30.641 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:30.652 INFO:teuthology.orchestra.run.smithi150.stdout:1643 trunc from 0xb4f5230 to 0x55ba18a 2024-09-21T23:10:30.654 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:10:30.699 INFO:teuthology.orchestra.run.smithi150.stdout:1644 writesame 0x9151dc0 thru 0x9161d20 data_size 0x2f(0xff61 bytes) 2024-09-21T23:10:30.699 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:30.697+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:30.706 INFO:teuthology.orchestra.run.smithi150.stdout:1645 trunc from 0x9161d21 to 0xdceba49 2024-09-21T23:10:30.717 INFO:teuthology.orchestra.run.smithi150.stdout:1646 punch from 0x207dac0 to 0x2081e91, (0x43d1 bytes) 2024-09-21T23:10:30.718 INFO:teuthology.orchestra.run.smithi150.stdout:1647 write 0x7198403 thru 0x71a68a2 (0xe4a0 bytes) 2024-09-21T23:10:30.723 INFO:teuthology.orchestra.run.smithi150.stdout:1649 read 0x921efad thru 0x92206ec (0x1740 bytes) 2024-09-21T23:10:30.736 INFO:teuthology.orchestra.run.smithi150.stdout:1650 trunc from 0xdceba49 to 0x5e3b17c 2024-09-21T23:10:30.757 INFO:teuthology.orchestra.run.smithi150.stdout:1651 read 0x2280116 thru 0x22846c5 (0x45b0 bytes) 2024-09-21T23:10:30.763 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:30.763 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.047344621999997, 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039594890000000001, 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716126+0000", 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716131+0000", 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716144+0000", 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2267e-05 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716158+0000", 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4731e-05 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716239+0000", 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1013999999999994e-05 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716271+0000", 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1535999999999998e-05 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720016+0000", 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037447399999999999 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720084+0000", 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8813999999999995e-05 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2850 3.4 e30/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 30'30, mlcod=30'28)", 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.737453199000001, 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033350860000000001, 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026017+0000", 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.68e-07 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026023+0000", 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1959999999999996e-06 2024-09-21T23:10:30.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026033+0000", 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8120000000000002e-06 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026042+0000", 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7059999999999993e-06 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026127+0000", 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4973000000000001e-05 2024-09-21T23:10:30.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026145+0000", 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8422000000000001e-05 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029303+0000", 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031578980000000001 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029351+0000", 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8411000000000001e-05 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.771 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:4287 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'16, mlcod=31'14)", 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.734101187, 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015395872, 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029371+0000", 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2759999999999999e-06 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031303+0000", 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019322359999999999 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031331+0000", 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8408999999999999e-05 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031342+0000", 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1086e-05 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031428+0000", 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5990999999999997e-05 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031454+0000", 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044695+0000", 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013240784 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044764+0000", 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9623999999999996e-05 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.775 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6012 3.4 e31/21 3:21794b57:::rbd_data.1149d3a069d9.0000000000000003:head v 31'34, mlcod=30'31)", 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.495077950999999, 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094147550000000003, 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268393+0000", 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.72e-06 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269210+0000", 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081656700000000003 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269235+0000", 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5000000000000001e-05 2024-09-21T23:10:30.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269248+0000", 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3393e-05 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269314+0000", 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6180000000000007e-05 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269341+0000", 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6554999999999999e-05 2024-09-21T23:10:30.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277723+0000", 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083823960000000003 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277806+0000", 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2943999999999997e-05 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6173 3.0 e32/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 32'22, mlcod=31'19)", 2024-09-21T23:10:30.779 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.165036064999999, 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002753003, 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598435+0000", 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8870000000000001e-06 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598445+0000", 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4949999999999997e-06 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598464+0000", 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8720999999999999e-05 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598473+0000", 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5980000000000003e-06 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598540+0000", 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7096999999999998e-05 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598564+0000", 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3288000000000001e-05 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601136+0000", 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025726099999999999 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601186+0000", 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0306999999999998e-05 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11716 3.9 3:94516a46:::rbd_data.1149e13ad964.000000000000006f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.833636725000002, 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097894799999999997, 2024-09-21T23:10:30.783 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11716 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929835+0000", 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2529999999999999e-06 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929846+0000", 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0859000000000001e-05 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929869+0000", 2024-09-21T23:10:30.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3450000000000001e-05 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929879+0000", 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7270000000000002e-06 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929973+0000", 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3917000000000002e-05 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.930106+0000", 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013346199999999999 2024-09-21T23:10:30.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939479+0000", 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.009372755 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939522+0000", 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3025999999999998e-05 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939622+0000", 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010003100000000001 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:12080 3.0 e36/21 3:003353e5:::rbd_data.1149e13ad964.000000000000003f:head v 36'38, mlcod=35'35)", 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.176382402000002, 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045448859999999997, 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587090+0000", 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.367e-06 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587109+0000", 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9816000000000001e-05 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587158+0000", 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8513999999999997e-05 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587179+0000", 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587395+0000", 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021613099999999999 2024-09-21T23:10:30.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587449+0000", 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3316000000000002e-05 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.591513+0000", 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040644160000000004 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.591632+0000", 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011884599999999999 2024-09-21T23:10:30.791 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21535 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.125532+0000", 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.637937387999999, 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022049029999999998, 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.792 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21535 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125532+0000", 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125532+0000", 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125535+0000", 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:30.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125551+0000", 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5483000000000001e-05 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125585+0000", 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4489999999999997e-05 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125602+0000", 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6529999999999999e-05 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125744+0000", 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014231199999999999 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.125909+0000", 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016517800000000001 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.127518+0000", 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016084719999999999 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.127615+0000", 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7134999999999997e-05 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.127737+0000", 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122099 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21566 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:30.796 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.498148203, 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002480384, 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21566 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:30.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265324+0000", 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9500000000000001e-06 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265341+0000", 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6303e-05 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265373+0000", 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2427e-05 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265408+0000", 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5126000000000002e-05 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265555+0000", 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014723500000000001 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265818+0000", 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026269800000000001 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267587+0000", 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017692000000000001 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267679+0000", 2024-09-21T23:10:30.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1922999999999994e-05 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267802+0000", 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012252199999999999 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21574 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.477463963, 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024147919999999998, 2024-09-21T23:10:30.801 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21574 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286008+0000", 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286021+0000", 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2714000000000001e-05 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286051+0000", 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0633000000000003e-05 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286082+0000", 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0213e-05 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286304+0000", 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000222112 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286570+0000", 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026670399999999999 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288119+0000", 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015483579999999999 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288208+0000", 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9133000000000002e-05 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288420+0000", 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021242900000000001 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21988 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.929190948, 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024888240000000002, 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21988 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834281+0000", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-06 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834294+0000", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2306e-05 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834323+0000", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.885e-05 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834337+0000", 2024-09-21T23:10:30.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4735999999999999e-05 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834488+0000", 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015039599999999999 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834745+0000", 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000257385 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836546+0000", 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018004220000000001 2024-09-21T23:10:30.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836638+0000", 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2483000000000002e-05 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836767+0000", 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012939900000000001 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.810 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:22023 3.0 e37/21 3:020038fe:::rbd_data.1149c115bf92.0000000000000003:head v 37'57, mlcod=37'55)", 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.931245+0000", 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.832224517, 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023235590000000002, 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931245+0000", 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931245+0000", 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931247+0000", 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0640000000000001e-06 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931341+0000", 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3795999999999994e-05 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931373+0000", 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1665999999999998e-05 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931386+0000", 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3482999999999999e-05 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931522+0000", 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013622600000000001 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.931574+0000", 2024-09-21T23:10:30.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2080000000000003e-05 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.933482+0000", 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019074280000000001 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.933569+0000", 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6816000000000004e-05 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:22077 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.696052583, 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025167950000000001, 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 22077 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067420+0000", 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7829999999999999e-06 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067434+0000", 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4116e-05 2024-09-21T23:10:30.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067467+0000", 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2790999999999998e-05 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067489+0000", 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067656+0000", 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016664100000000001 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067913+0000", 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000257154 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069732+0000", 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001819036 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069820+0000", 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7590000000000007e-05 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069934+0000", 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011401200000000001 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:22098 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.187051671000001, 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066791929999999999, 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 22098 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576421+0000", 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6090000000000001e-06 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576439+0000", 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7802e-05 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576481+0000", 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1653999999999998e-05 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576507+0000", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6001000000000001e-05 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576679+0000", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017189700000000001 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576707+0000", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8608000000000001e-05 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.580886+0000", 2024-09-21T23:10:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041785920000000001 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.581057+0000", 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000170915 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.582894+0000", 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018374070000000001 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.582981+0000", 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6436999999999999e-05 2024-09-21T23:10:30.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.583097+0000", 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116271 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29041 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8002125910000002, 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024231119999999998, 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29041 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963260+0000", 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.224e-06 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963272+0000", 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1817999999999999e-05 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963301+0000", 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9268000000000002e-05 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963315+0000", 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3756e-05 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963483+0000", 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167863 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963752+0000", 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026917 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965465+0000", 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001713259 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965559+0000", 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3521000000000001e-05 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965680+0000", 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121233 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29116 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.6628954450000002, 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024782770000000001, 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29116 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100577+0000", 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100590+0000", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2959000000000001e-05 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100620+0000", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9833e-05 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100634+0000", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4167e-05 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100794+0000", 2024-09-21T23:10:30.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015995500000000001 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.101056+0000", 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026250900000000002 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.102829+0000", 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001772259 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.102924+0000", 2024-09-21T23:10:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4874e-05 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.103052+0000", 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000128856 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29126 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.115287+0000", 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.6481827989999998, 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002371293, 2024-09-21T23:10:30.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29126 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115287+0000", 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115287+0000", 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115290+0000", 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0989999999999999e-06 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115302+0000", 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.198e-05 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115332+0000", 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0558000000000001e-05 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115346+0000", 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3925e-05 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115507+0000", 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016117099999999999 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.115776+0000", 2024-09-21T23:10:30.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026852700000000001 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.117491+0000", 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001715428 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.117559+0000", 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8065999999999995e-05 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.117658+0000", 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539e-05 2024-09-21T23:10:30.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29244 3.9 3:95c8871e:::rbd_data.11499299167c.0000000000000302:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.4568663449999999, 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080285210000000003, 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29244 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306606+0000", 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306620+0000", 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3978999999999999e-05 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306642+0000", 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2557000000000001e-05 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306655+0000", 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2894000000000001e-05 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306715+0000", 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9485999999999999e-05 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306804+0000", 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9603000000000003e-05 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314472+0000", 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076677029999999997 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314552+0000", 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0275e-05 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314632+0000", 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9599999999999997e-05 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29590 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.1362002909999998, 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058105209999999999, 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29590 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.843 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627273+0000", 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7069999999999998e-06 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627288+0000", 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5129e-05 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627322+0000", 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3494999999999997e-05 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627347+0000", 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5191e-05 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627458+0000", 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011141000000000001 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627467+0000", 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.630921+0000", 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00345467 2024-09-21T23:10:30.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.631080+0000", 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015890500000000001 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.632864+0000", 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017839729999999999 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.632961+0000", 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6784e-05 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.633080+0000", 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001189 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:33981 3.9 3:9114e073:::rbd_data.114937d450a8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:30.362164+0000", 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.40130534800000001, 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027857300000000002, 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:30.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 33981 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362164+0000", 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362164+0000", 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362167+0000", 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4509999999999999e-06 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362527+0000", 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036011300000000001 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362547+0000", 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0239000000000001e-05 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362556+0000", 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4839999999999993e-06 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362630+0000", 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3718000000000001e-05 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362767+0000", 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013664999999999999 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.364535+0000", 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017677859999999999 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.364711+0000", 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017594400000000001 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.364950+0000", 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023934499999999999 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:30.853 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:30.853 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:30.853 INFO:teuthology.orchestra.run.smithi150.stdout:1652 write 0x6750cde thru 0x6756a16 (0x5d39 bytes) 2024-09-21T23:10:30.854 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:30.855 INFO:teuthology.orchestra.run.smithi150.stdout:1655 writesame 0x8dc0db1 thru 0x8dce547 data_size 0x1(0xd797 bytes) 2024-09-21T23:10:30.855 INFO:teuthology.orchestra.run.smithi150.stdout:1656 write 0x6dfc514 thru 0x6e04d63 (0x8850 bytes) 2024-09-21T23:10:30.855 INFO:teuthology.orchestra.run.smithi150.stdout:1660 trunc from 0x8dce548 to 0xa39a6f2 2024-09-21T23:10:30.855 INFO:teuthology.orchestra.run.smithi150.stdout:1661 trunc from 0xa39a6f2 to 0x1929566 2024-09-21T23:10:30.855 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:30.853+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:30.858 INFO:teuthology.orchestra.run.smithi150.stdout:1662 read 0x140da83 thru 0x140e4cb (0xa49 bytes) 2024-09-21T23:10:30.860 INFO:teuthology.orchestra.run.smithi150.stdout:1664 read 0x44d93 thru 0x4bc71 (0x6edf bytes) 2024-09-21T23:10:30.893 INFO:teuthology.orchestra.run.smithi150.stdout:1666 write 0x9c543bf thru 0x9c58f67 (0x4ba9 bytes) 2024-09-21T23:10:30.897 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:30.898 INFO:teuthology.orchestra.run.smithi150.stdout:1668 read 0x9a67d5 thru 0x9b6015 (0xf841 bytes) 2024-09-21T23:10:30.898 INFO:teuthology.orchestra.run.smithi150.stdout:1669 punch from 0x920709a to 0x92132db, (0xc241 bytes) 2024-09-21T23:10:30.919 INFO:teuthology.orchestra.run.smithi150.stdout:1671 write 0xed90d53 thru 0xed9f95c (0xec0a bytes) 2024-09-21T23:10:30.925 INFO:teuthology.orchestra.run.smithi150.stdout:1673 write 0x80d83e4 thru 0x80da8da (0x24f7 bytes) 2024-09-21T23:10:30.938 INFO:teuthology.orchestra.run.smithi150.stdout:1674 trunc from 0xed9f95d to 0xb15284c 2024-09-21T23:10:30.955 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:30.953+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:30.962 INFO:teuthology.orchestra.run.smithi150.stdout:1676 punch from 0x1be63f9 to 0x1beeaac, (0x86b3 bytes) 2024-09-21T23:10:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:1677 writesame 0xc6a09b6 thru 0xc6a7ae8 data_size 0x1(0x7133 bytes) 2024-09-21T23:10:31.000 INFO:teuthology.orchestra.run.smithi150.stdout:1678 trunc from 0xc6a7ae9 to 0x16a6bb7 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:34238 3.1e 3:7c714047:::rbd_object_map.114937d450a8:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:31.008854+0000", 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0033196670000000001, 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033709679999999998, 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 34238 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.008854+0000", 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.008854+0000", 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.008855+0000", 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.008861+0000", 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9540000000000001e-06 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.008877+0000", 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6140000000000001e-05 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.008888+0000", 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0657e-05 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.009008+0000", 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120097 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.009274+0000", 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026616699999999999 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:10:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.010304+0000", 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001029259 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.011713+0000", 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014094369999999999 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 1 2024-09-21T23:10:31.015 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:31.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:1679 write 0xb79c906 thru 0xb7ab7f0 (0xeeeb bytes) 2024-09-21T23:10:31.055 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:31.053+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:31.063 INFO:teuthology.orchestra.run.smithi150.stdout:1680 trunc from 0xb7ab7f1 to 0xe66d0fa 2024-09-21T23:10:31.074 INFO:teuthology.orchestra.run.smithi150.stdout:1681 writesame 0x215bec thru 0x21b55b data_size 0x6c(0x5970 bytes) 2024-09-21T23:10:31.077 INFO:teuthology.orchestra.run.smithi150.stdout:1682 write 0xe855df thru 0xe9398f (0xe3b1 bytes) 2024-09-21T23:10:31.079 INFO:teuthology.orchestra.run.smithi150.stdout:1685 read 0xc8a5221 thru 0xc8a8c9a (0x3a7a bytes) 2024-09-21T23:10:31.103 INFO:teuthology.orchestra.run.smithi150.stdout:1688 trunc from 0xe66d0fa to 0xcd1b97 2024-09-21T23:10:31.137 INFO:teuthology.orchestra.run.smithi150.stdout:1690 write 0x14ffa32 thru 0x150108e (0x165d bytes) 2024-09-21T23:10:31.156 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:31.153+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:31.165 INFO:teuthology.orchestra.run.smithi150.stdout:1691 write 0x8948223 thru 0x89548cd (0xc6ab bytes) 2024-09-21T23:10:31.172 INFO:teuthology.orchestra.run.smithi150.stdout:1694 punch from 0x1a19a00 to 0x1a223e9, (0x89e9 bytes) 2024-09-21T23:10:31.178 INFO:teuthology.orchestra.run.smithi150.stdout:1695 trunc from 0x89548ce to 0xbe7a3e6 2024-09-21T23:10:31.188 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:31.189 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:31.189 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:31.191 INFO:teuthology.orchestra.run.smithi150.stdout:1696 write 0x19f8226 thru 0x1a015d2 (0x93ad bytes) 2024-09-21T23:10:31.195 INFO:teuthology.orchestra.run.smithi150.stdout:1697 read 0xb0812a6 thru 0xb08f837 (0xe592 bytes) 2024-09-21T23:10:31.207 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:31.213 INFO:teuthology.orchestra.run.smithi150.stdout:1698 writesame 0xec2684e thru 0xec2e618 data_size 0x1(0x7dcb bytes) 2024-09-21T23:10:31.213 INFO:teuthology.orchestra.run.smithi150.stdout:1700 write 0x5e2a733 thru 0x5e33688 (0x8f56 bytes) 2024-09-21T23:10:31.220 INFO:teuthology.orchestra.run.smithi150.stdout:1701 read 0x22fd589 thru 0x22fed33 (0x17ab bytes) 2024-09-21T23:10:31.220 INFO:teuthology.orchestra.run.smithi150.stdout:1702 writesame 0xaab3928 thru 0xaabd440 data_size 0x5(0x9b19 bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1703 punch from 0x1b642fd to 0x1b6c06d, (0x7d70 bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1704 punch from 0x1e7a078 to 0x1e83057, (0x8fdf bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1709 read 0x8546464 thru 0x8547ca2 (0x183f bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1713 read 0xc2bccf6 thru 0xc2be628 (0x1933 bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1714 read 0x546589f thru 0x5471a46 (0xc1a8 bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1715 punch from 0x7affac9 to 0x7b0006f, (0x5a6 bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1716 read 0x8407590 thru 0x8416f10 (0xf981 bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1718 read 0x522ea6a thru 0x523c405 (0xd99c bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1719 read 0xa29d721 thru 0xa2a0917 (0x31f7 bytes) 2024-09-21T23:10:31.231 INFO:teuthology.orchestra.run.smithi150.stdout:1722 read 0x6749bca thru 0x674b7cc (0x1c03 bytes) 2024-09-21T23:10:31.232 INFO:teuthology.orchestra.run.smithi150.stdout:1723 punch from 0x5deda9f to 0x5df54fc, (0x7a5d bytes) 2024-09-21T23:10:31.232 INFO:teuthology.orchestra.run.smithi150.stdout:1725 punch from 0x32aeb8d to 0x32bc995, (0xde08 bytes) 2024-09-21T23:10:31.232 INFO:teuthology.orchestra.run.smithi150.stdout:1726 read 0x27d0d0 thru 0x289fa7 (0xced8 bytes) 2024-09-21T23:10:31.232 INFO:teuthology.orchestra.run.smithi150.stdout:1731 write 0x8aa6fec thru 0x8ab555a (0xe56f bytes) 2024-09-21T23:10:31.236 INFO:teuthology.orchestra.run.smithi150.stdout:1732 punch from 0x2face2a to 0x2fbb8bf, (0xea95 bytes) 2024-09-21T23:10:31.236 INFO:teuthology.orchestra.run.smithi150.stdout:1733 read 0xd2aa963 thru 0xd2ba283 (0xf921 bytes) 2024-09-21T23:10:31.236 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.236 INFO:teuthology.orchestra.run.smithi150.stdout:1735 writesame 0x39c8d0a thru 0x39d770a data_size 0x1(0xea01 bytes) 2024-09-21T23:10:31.257 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:31.257+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:31.273 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.273 INFO:teuthology.orchestra.run.smithi150.stdout:1737 read 0x6e4aa7e thru 0x6e59a2d (0xefb0 bytes) 2024-09-21T23:10:31.273 INFO:teuthology.orchestra.run.smithi150.stdout:1738 punch from 0xac14681 to 0xac17b03, (0x3482 bytes) 2024-09-21T23:10:31.273 INFO:teuthology.orchestra.run.smithi150.stdout:1739 writesame 0x3a2046c thru 0x3a2f121 data_size 0x29(0xecb6 bytes) 2024-09-21T23:10:31.281 INFO:teuthology.orchestra.run.smithi150.stdout:1740 read 0xe229ee0 thru 0xe22b3c9 (0x14ea bytes) 2024-09-21T23:10:31.281 INFO:teuthology.orchestra.run.smithi150.stdout:1742 write 0x2d513f6 thru 0x2d517fd (0x408 bytes) 2024-09-21T23:10:31.284 INFO:teuthology.orchestra.run.smithi150.stdout:1744 read 0x5d52d90 thru 0x5d622e3 (0xf554 bytes) 2024-09-21T23:10:31.284 INFO:teuthology.orchestra.run.smithi150.stdout:1746 write 0xc58663 thru 0xc637aa (0xb148 bytes) 2024-09-21T23:10:31.286 INFO:teuthology.orchestra.run.smithi150.stdout:1748 writesame 0x532abc2 thru 0x5338237 data_size 0x61(0xd676 bytes) 2024-09-21T23:10:31.288 INFO:teuthology.orchestra.run.smithi150.stdout:1750 writesame 0x2805bae thru 0x280fb1f data_size 0x3(0x9f72 bytes) 2024-09-21T23:10:31.293 INFO:teuthology.orchestra.run.smithi150.stdout:1751 write 0xe5dfd1c thru 0xe5eb0bd (0xb3a2 bytes) 2024-09-21T23:10:31.300 INFO:teuthology.orchestra.run.smithi150.stdout:1752 write 0x51bd2a8 thru 0x51bfe34 (0x2b8d bytes) 2024-09-21T23:10:31.304 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.304 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.304 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.304 INFO:teuthology.orchestra.run.smithi150.stdout:1759 read 0x20dae7c thru 0x20e13f6 (0x657b bytes) 2024-09-21T23:10:31.305 INFO:teuthology.orchestra.run.smithi150.stdout:1760 read 0x275edff thru 0x27697a9 (0xa9ab bytes) 2024-09-21T23:10:31.305 INFO:teuthology.orchestra.run.smithi150.stdout:1761 write 0x8193eb0 thru 0x81969e2 (0x2b33 bytes) 2024-09-21T23:10:31.310 INFO:teuthology.orchestra.run.smithi150.stdout:1762 write 0x8f18b6c thru 0x8f24995 (0xbe2a bytes) 2024-09-21T23:10:31.316 INFO:teuthology.orchestra.run.smithi150.stdout:1764 read 0x794f08c thru 0x7957e89 (0x8dfe bytes) 2024-09-21T23:10:31.316 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.316 INFO:teuthology.orchestra.run.smithi150.stdout:1767 write 0x29bae7a thru 0x29bb911 (0xa98 bytes) 2024-09-21T23:10:31.318 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.318 INFO:teuthology.orchestra.run.smithi150.stdout:1770 writesame 0xda0d73b thru 0xda12ce8 data_size 0xb(0x55ae bytes) 2024-09-21T23:10:31.331 INFO:teuthology.orchestra.run.smithi150.stdout:1772 trunc from 0xec2e619 to 0xa18505a 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4902 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:31.351 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.171789097000001, 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030793945, 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4902 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:31.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.37e-07 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179183+0000", 2024-09-21T23:10:31.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9280000000000002e-06 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179192+0000", 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7020000000000007e-06 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179198+0000", 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6780000000000003e-06 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201407+0000", 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022208301999999999 2024-09-21T23:10:31.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201500+0000", 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3231000000000002e-05 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.209974+0000", 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084735669999999996 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4903 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2359296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.171761121999999, 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031240032000000001, 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4903 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179211+0000", 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6630000000000002e-06 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179220+0000", 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8480000000000007e-06 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179226+0000", 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2650000000000002e-06 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210016+0000", 2024-09-21T23:10:31.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030790128 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210066+0000", 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9713000000000003e-05 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210448+0000", 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038190499999999998 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.359 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4904 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.171733894999999, 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031636297000000001, 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4904 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179238+0000", 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5349999999999999e-06 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179247+0000", 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6449999999999994e-06 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179260+0000", 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3431999999999999e-05 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210480+0000", 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031220121 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210550+0000", 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0352000000000007e-05 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210871+0000", 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000320701 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4905 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.171698729999999, 2024-09-21T23:10:31.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036964023999999998, 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4905 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179271+0000", 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179274+0000", 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1350000000000001e-06 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179283+0000", 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1779999999999992e-06 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179290+0000", 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210903+0000", 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031613162 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210976+0000", 2024-09-21T23:10:31.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2591999999999999e-05 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216234+0000", 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052583309999999998 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4906 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.171668987, 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037394366999999998, 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.367 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4906 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:31.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179303+0000", 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8219999999999998e-06 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179312+0000", 2024-09-21T23:10:31.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6419999999999996e-06 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179319+0000", 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6449999999999999e-06 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216272+0000", 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036953142000000001 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216352+0000", 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0044999999999997e-05 2024-09-21T23:10:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216694+0000", 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034238399999999999 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5226 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.158272588999999, 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029942795000000001, 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5226 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:31.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192697+0000", 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7199999999999998e-07 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192701+0000", 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8e-06 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192712+0000", 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1156e-05 2024-09-21T23:10:31.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192724+0000", 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2551999999999999e-05 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222110+0000", 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029385620000000001 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222191+0000", 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1520000000000006e-05 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222639+0000", 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044747500000000002 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5227 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.158233248999998, 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030372671, 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5227 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192736+0000", 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192742+0000", 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2260000000000002e-06 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192753+0000", 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0762e-05 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192761+0000", 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.14e-06 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222674+0000", 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029912979999999999 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222754+0000", 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9816999999999998e-05 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223108+0000", 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035405799999999998 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5228 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.158196335, 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0320895, 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5228 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192773+0000", 2024-09-21T23:10:31.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192776+0000", 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2380000000000002e-06 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192787+0000", 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0764999999999999e-05 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192795+0000", 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3000000000000002e-06 2024-09-21T23:10:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223154+0000", 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030358672999999999 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223230+0000", 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6268999999999998e-05 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224862+0000", 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001631635 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5229 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.158162151999999, 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032496191000000001, 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5229 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5199999999999996e-07 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192810+0000", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192820+0000", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0132000000000001e-05 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192829+0000", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5900000000000008e-06 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224912+0000", 2024-09-21T23:10:31.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032083410999999999 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224985+0000", 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2455999999999999e-05 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225303+0000", 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031788999999999998 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5230 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.158127478000001, 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033015543000000001, 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5230 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192842+0000", 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192845+0000", 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:31.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192856+0000", 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1008e-05 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192865+0000", 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225398+0000", 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032533617000000001 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225489+0000", 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1038999999999994e-05 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225857+0000", 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000367597 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5231 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:31.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.158093556000001, 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033349594000000003, 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5231 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192876+0000", 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192879+0000", 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2059999999999999e-06 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192890+0000", 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0702e-05 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192898+0000", 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3299999999999999e-06 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225903+0000", 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033004622999999997 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225981+0000", 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7990000000000004e-05 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226225+0000", 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024411499999999999 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5232 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3801088~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:31.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.158059586, 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033795809000000003, 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5232 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192910+0000", 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1800000000000005e-07 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192913+0000", 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1669999999999999e-06 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192924+0000", 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0692e-05 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192936+0000", 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2414e-05 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226252+0000", 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03331626 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226309+0000", 2024-09-21T23:10:31.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6733e-05 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226705+0000", 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00039582499999999999 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5406 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.152546338000001, 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029727338999999998, 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5406 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198423+0000", 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7100000000000001e-07 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198427+0000", 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198436+0000", 2024-09-21T23:10:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198446+0000", 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6360000000000006e-06 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227686+0000", 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029240289999999999 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227764+0000", 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7676000000000006e-05 2024-09-21T23:10:31.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228150+0000", 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038574300000000002 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5407 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.152512460000001, 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030154378999999999, 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5407 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198457+0000", 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198460+0000", 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3409999999999999e-06 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198470+0000", 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:31.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198478+0000", 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7130000000000005e-06 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228216+0000", 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029738243000000001 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228290+0000", 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3461999999999996e-05 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228611+0000", 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032108100000000002 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5408 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.152480994000001, 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030920828000000001, 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5408 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198491+0000", 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198501+0000", 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5719999999999992e-06 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198508+0000", 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2470000000000002e-06 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228720+0000", 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030211598999999999 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228851+0000", 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013098299999999999 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229409+0000", 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055781799999999996 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5412 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.150375985, 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030113958, 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5412 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4099999999999998e-07 2024-09-21T23:10:31.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200626+0000", 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2672999999999997e-05 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200637+0000", 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1248e-05 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200646+0000", 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6559999999999998e-06 2024-09-21T23:10:31.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230319+0000", 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029672765 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230378+0000", 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8696999999999997e-05 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230707+0000", 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032917800000000001 2024-09-21T23:10:31.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.276282740999999, 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:31.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:31.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:31.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.110515889999999, 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:31.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:31.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:21897 3.5 e37/21 3:a0a2d2ee:::rbd_data.1149c115bf92.0000000000000000:head v 37'78, mlcod=37'76)", 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.858792717, 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037373174000000002, 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492178+0000", 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9400000000000001e-06 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499469+0000", 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072913370000000002 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499506+0000", 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6412999999999998e-05 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499524+0000", 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8097e-05 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499683+0000", 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000159631 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499726+0000", 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2716999999999998e-05 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.529473+0000", 2024-09-21T23:10:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029746356000000002 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.529549+0000", 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6682999999999997e-05 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:27646 3.b 3:d396a686:::rbd_data.11499b8b109.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.9952488989999999, 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032669003000000002, 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:31.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 27646 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:31.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355723+0000", 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.360977+0000", 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052540290000000003 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361008+0000", 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0840000000000003e-05 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361028+0000", 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9771000000000001e-05 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361182+0000", 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015453799999999999 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361827+0000", 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064513600000000004 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.363660+0000", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001833131 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388259+0000", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024598811000000002 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388305+0000", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5479000000000003e-05 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388341+0000", 2024-09-21T23:10:31.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6009e-05 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388389+0000", 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8229000000000002e-05 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:31.433 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:31.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:31.434 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.434 INFO:teuthology.orchestra.run.smithi150.stdout:1774 write 0x81a60a8 thru 0x81ad009 (0x6f62 bytes) 2024-09-21T23:10:31.434 INFO:teuthology.orchestra.run.smithi150.stdout:1775 write 0x254af49 thru 0x2557dd0 (0xce88 bytes) 2024-09-21T23:10:31.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:31.436 INFO:teuthology.orchestra.run.smithi150.stdout:1776 write 0xec32e15 thru 0xec338ea (0xad6 bytes) 2024-09-21T23:10:31.436 INFO:teuthology.orchestra.run.smithi150.stdout:1777 read 0x90a6db7 thru 0x90a85ca (0x1814 bytes) 2024-09-21T23:10:31.436 INFO:teuthology.orchestra.run.smithi150.stdout:1778 trunc from 0xec338eb to 0x4759d5 2024-09-21T23:10:31.436 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:31.433+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:31.482 INFO:teuthology.orchestra.run.smithi150.stdout:1779 write 0x8991170 thru 0x8991f08 (0xd99 bytes) 2024-09-21T23:10:31.499 INFO:teuthology.orchestra.run.smithi150.stdout:1780 trunc from 0x8991f09 to 0xec7d96f 2024-09-21T23:10:31.509 INFO:teuthology.orchestra.run.smithi150.stdout:1782 writesame 0x5446977 thru 0x544bd1d data_size 0x1(0x53a7 bytes) 2024-09-21T23:10:31.536 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:31.537+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:31.542 INFO:teuthology.orchestra.run.smithi150.stdout:1783 write 0x5d63e00 thru 0x5d723e2 (0xe5e3 bytes) 2024-09-21T23:10:31.548 INFO:teuthology.orchestra.run.smithi150.stdout:1784 write 0x138eadf thru 0x13996b9 (0xabdb bytes) 2024-09-21T23:10:31.555 INFO:teuthology.orchestra.run.smithi150.stdout:1785 read 0xb3f3ca2 thru 0xb3ff200 (0xb55f bytes) 2024-09-21T23:10:31.555 INFO:teuthology.orchestra.run.smithi150.stdout:1787 writesame 0x9bbb46e thru 0x9bc78fd data_size 0x55(0xc490 bytes) 2024-09-21T23:10:31.570 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:31.570 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:31.574 INFO:teuthology.orchestra.run.smithi150.stdout:1790 trunc from 0xec7d96f to 0x61c8136 2024-09-21T23:10:31.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:31.595 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.605 INFO:teuthology.orchestra.run.smithi150.stdout:1792 trunc from 0x61c8136 to 0x70c98b 2024-09-21T23:10:31.627 INFO:teuthology.orchestra.run.smithi150.stdout:1794 trunc from 0x70c98b to 0x4c9ae6a 2024-09-21T23:10:31.634 INFO:teuthology.orchestra.run.smithi150.stdout:1795 punch from 0x147e640 to 0x147f0a7, (0xa67 bytes) 2024-09-21T23:10:31.636 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:31.637+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:31.642 INFO:teuthology.orchestra.run.smithi150.stdout:1796 trunc from 0x4c9ae6a to 0xa259b12 2024-09-21T23:10:31.649 INFO:teuthology.orchestra.run.smithi150.stdout:1797 write 0x4b9195e thru 0x4b92025 (0x6c8 bytes) 2024-09-21T23:10:31.652 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.652 INFO:teuthology.orchestra.run.smithi150.stdout:1800 writesame 0x3f6eda2 thru 0x3f7216d data_size 0x55(0x33cc bytes) 2024-09-21T23:10:31.653 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:31.653 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:31.656 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.656 INFO:teuthology.orchestra.run.smithi150.stdout:1802 punch from 0x576ea15 to 0x5771706, (0x2cf1 bytes) 2024-09-21T23:10:31.659 INFO:teuthology.orchestra.run.smithi150.stdout:1804 trunc from 0xa259b12 to 0xc7eb9ed 2024-09-21T23:10:31.666 INFO:teuthology.orchestra.run.smithi150.stdout:1805 write 0x7f62f5b thru 0x7f660ca (0x3170 bytes) 2024-09-21T23:10:31.669 INFO:teuthology.orchestra.run.smithi150.stdout:1807 writesame 0x8ce37c7 thru 0x8cec0c5 data_size 0x1(0x88ff bytes) 2024-09-21T23:10:31.670 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:31.670 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:31.687 INFO:teuthology.orchestra.run.smithi150.stdout:1809 write 0xaddf9 thru 0xb1f7c (0x4184 bytes) 2024-09-21T23:10:31.687 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:31.687 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:31.694 INFO:teuthology.orchestra.run.smithi150.stdout:1810 trunc from 0xc7eb9ed to 0x93b6fbf 2024-09-21T23:10:31.704 INFO:teuthology.orchestra.run.smithi150.stdout:1811 punch from 0xc255f to 0xc6206, (0x3ca7 bytes) 2024-09-21T23:10:31.704 INFO:teuthology.orchestra.run.smithi150.stdout:1812 write 0x4160196 thru 0x416ff35 (0xfda0 bytes) 2024-09-21T23:10:31.706 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:31.706 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:31.706 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:31.708 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.708 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.708 INFO:teuthology.orchestra.run.smithi150.stdout:1815 read 0xcdc375 thru 0xce3a30 (0x76bc bytes) 2024-09-21T23:10:31.708 INFO:teuthology.orchestra.run.smithi150.stdout:1816 read 0x83c5d71 thru 0x83cff65 (0xa1f5 bytes) 2024-09-21T23:10:31.708 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.708 INFO:teuthology.orchestra.run.smithi150.stdout:1818 read 0x404b122 thru 0x404faaa (0x4989 bytes) 2024-09-21T23:10:31.708 INFO:teuthology.orchestra.run.smithi150.stdout:1819 write 0x837cd05 thru 0x8389162 (0xc45e bytes) 2024-09-21T23:10:31.712 INFO:teuthology.orchestra.run.smithi150.stdout:1820 read 0x8448755 thru 0x8448a77 (0x323 bytes) 2024-09-21T23:10:31.713 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:31.713 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:31.713 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:31.713 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:31.713 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:31.726 INFO:teuthology.orchestra.run.smithi150.stdout:1821 write 0xc72949c thru 0xc72c557 (0x30bc bytes) 2024-09-21T23:10:31.727 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:31.728 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:31.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:31.730 INFO:teuthology.orchestra.run.smithi150.stdout:1823 punch from 0xbe8d83a to 0xbe971c4, (0x998a bytes) 2024-09-21T23:10:31.730 INFO:teuthology.orchestra.run.smithi150.stdout:1824 writesame 0xa4807ba thru 0xa486031 data_size 0x13(0x5878 bytes) 2024-09-21T23:10:31.738 INFO:teuthology.orchestra.run.smithi150.stdout:1825 read 0x9ba41c0 thru 0x9bad1ef (0x9030 bytes) 2024-09-21T23:10:31.738 INFO:teuthology.orchestra.run.smithi150.stdout:1826 read 0x86b7d19 thru 0x86b96f2 (0x19da bytes) 2024-09-21T23:10:31.738 INFO:teuthology.orchestra.run.smithi150.stdout:1827 write 0x200f7c thru 0x2077b6 (0x683b bytes) 2024-09-21T23:10:31.738 INFO:teuthology.orchestra.run.smithi150.stdout:1830 read 0x9c9f619 thru 0x9ca6838 (0x7220 bytes) 2024-09-21T23:10:31.739 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:31.737+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:31.747 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:31.747 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:31.748 INFO:teuthology.orchestra.run.smithi150.stdout:1831 write 0xd6aed70 thru 0xd6bc669 (0xd8fa bytes) 2024-09-21T23:10:31.751 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.751 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:31.751 INFO:teuthology.orchestra.run.smithi150.stdout:1834 write 0x4d1a389 thru 0x4d1f02a (0x4ca2 bytes) 2024-09-21T23:10:31.755 INFO:teuthology.orchestra.run.smithi150.stdout:1836 trunc from 0xd6bc66a to 0xe175aa3 2024-09-21T23:10:31.760 INFO:teuthology.orchestra.run.smithi150.stdout:1837 clone 19 order 25 su 4194304 sc 10 2024-09-21T23:10:31.765 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:31.765 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:31.769 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:31.770 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:31.839 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:31.841+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:31.855 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:31.855 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:31.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:31.940 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:31.937+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:31.992 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:31.992 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:32.006 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:32.041 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:32.041+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:32.142 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:32.141+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:32.172 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:32.172 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:32.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:32.242 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:32.240+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:32.305 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:32.305 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:32.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:32.343 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:32.341+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:32.444 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:32.445+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:32.486 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:32.486 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:32.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:32.545 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:32.545+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:32.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:32.647 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:32.645+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:32.687 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:10:32.747 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:32.744+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:32.766 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone18 from 0xe175aa3 (overlap 0x21a95a) to 0xd76f0 2024-09-21T23:10:32.766 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:32.766 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:32.785 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:32.848 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:32.844+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:32.853 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-21T23:10:32.859 INFO:teuthology.orchestra.run.smithi150.stdout:1839 read 0x476c37 thru 0x47ecf6 (0x80c0 bytes) 2024-09-21T23:10:32.860 INFO:teuthology.orchestra.run.smithi150.stdout:1840 writesame 0x2571d47 thru 0x257cb49 data_size 0x1f(0xae03 bytes) 2024-09-21T23:10:32.872 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:32.872 INFO:teuthology.orchestra.run.smithi150.stdout:1845 read 0x3dc95c7 thru 0x3dcf7c7 (0x6201 bytes) 2024-09-21T23:10:32.872 INFO:teuthology.orchestra.run.smithi150.stdout:1847 write 0x7063a7b thru 0x706989c (0x5e22 bytes) 2024-09-21T23:10:32.877 INFO:teuthology.orchestra.run.smithi150.stdout:1849 punch from 0x2a445fc to 0x2a4d2c5, (0x8cc9 bytes) 2024-09-21T23:10:32.877 INFO:teuthology.orchestra.run.smithi150.stdout:1851 read 0x8513c44 thru 0x8513fa9 (0x366 bytes) 2024-09-21T23:10:32.877 INFO:teuthology.orchestra.run.smithi150.stdout:1852 write 0x7a06eda thru 0x7a13e44 (0xcf6b bytes) 2024-09-21T23:10:32.892 INFO:teuthology.orchestra.run.smithi150.stdout:1853 trunc from 0xe175aa3 to 0xdd5514f 2024-09-21T23:10:32.908 INFO:teuthology.orchestra.run.smithi150.stdout:1855 read 0x373a9fc thru 0x3746232 (0xb837 bytes) 2024-09-21T23:10:32.908 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:32.908 INFO:teuthology.orchestra.run.smithi150.stdout:1857 read 0x8b15f54 thru 0x8b1a6f4 (0x47a1 bytes) 2024-09-21T23:10:32.908 INFO:teuthology.orchestra.run.smithi150.stdout:1858 punch from 0x1dd7aa9 to 0x1ddb901, (0x3e58 bytes) 2024-09-21T23:10:32.908 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:32.908 INFO:teuthology.orchestra.run.smithi150.stdout:1861 write 0x67829b9 thru 0x678f234 (0xc87c bytes) 2024-09-21T23:10:32.909 INFO:teuthology.orchestra.run.smithi150.stdout:1863 punch from 0xd583f7c to 0xd588242, (0x42c6 bytes) 2024-09-21T23:10:32.917 INFO:teuthology.orchestra.run.smithi150.stdout:1864 write 0xe85f1b6 thru 0xe86ce34 (0xdc7f bytes) 2024-09-21T23:10:32.919 INFO:teuthology.orchestra.run.smithi150.stdout:1866 read 0x3a72cbc thru 0x3a7e94b (0xbc90 bytes) 2024-09-21T23:10:32.919 INFO:teuthology.orchestra.run.smithi150.stdout:1867 punch from 0x3d1c149 to 0x3d28e2d, (0xcce4 bytes) 2024-09-21T23:10:32.919 INFO:teuthology.orchestra.run.smithi150.stdout:1868 punch from 0x5088132 to 0x5097552, (0xf420 bytes) 2024-09-21T23:10:32.919 INFO:teuthology.orchestra.run.smithi150.stdout:1869 punch from 0x831216b to 0x8320dcb, (0xec60 bytes) 2024-09-21T23:10:32.919 INFO:teuthology.orchestra.run.smithi150.stdout:1870 write 0x9309f12 thru 0x930a517 (0x606 bytes) 2024-09-21T23:10:32.920 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:32.920 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:32.921 INFO:teuthology.orchestra.run.smithi150.stdout:1871 read 0xe18893b thru 0xe1953a4 (0xca6a bytes) 2024-09-21T23:10:32.922 INFO:teuthology.orchestra.run.smithi150.stdout:1872 writesame 0xd8f2ba5 thru 0xd8f7372 data_size 0x5b(0x47ce bytes) 2024-09-21T23:10:32.929 INFO:teuthology.orchestra.run.smithi150.stdout:1873 writesame 0xdb1a7f1 thru 0xdb23d95 data_size 0x1(0x95a5 bytes) 2024-09-21T23:10:32.936 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:32.980 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:32.980+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:32.983 INFO:teuthology.orchestra.run.smithi150.stdout:1874 trunc from 0xe86ce35 to 0xb51be8a 2024-09-21T23:10:33.001 INFO:teuthology.orchestra.run.smithi150.stdout:1875 write 0x1d9074f thru 0x1d9d5d8 (0xce8a bytes) 2024-09-21T23:10:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:1876 writesame 0xbac1a86 thru 0xbacf363 data_size 0x39(0xd8de bytes) 2024-09-21T23:10:33.013 INFO:teuthology.orchestra.run.smithi150.stdout:1878 write 0x330f669 thru 0x331db38 (0xe4d0 bytes) 2024-09-21T23:10:33.015 INFO:teuthology.orchestra.run.smithi150.stdout:1879 read 0x2467a10 thru 0x246c360 (0x4951 bytes) 2024-09-21T23:10:33.016 INFO:teuthology.orchestra.run.smithi150.stdout:1880 read 0x6e768a3 thru 0x6e784c9 (0x1c27 bytes) 2024-09-21T23:10:33.017 INFO:teuthology.orchestra.run.smithi150.stdout:1881 read 0x7fda2e6 thru 0x7fdc219 (0x1f34 bytes) 2024-09-21T23:10:33.033 INFO:teuthology.orchestra.run.smithi150.stdout:1882 trunc from 0xbacf364 to 0x1840480 2024-09-21T23:10:33.049 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:33.048+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:33.069 INFO:teuthology.orchestra.run.smithi150.stdout:1883 write 0x9ebc76e thru 0x9ec8804 (0xc097 bytes) 2024-09-21T23:10:33.073 INFO:teuthology.orchestra.run.smithi150.stdout:1884 read 0x1e2ede1 thru 0x1e3a6e1 (0xb901 bytes) 2024-09-21T23:10:33.073 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:33.073 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:33.074 INFO:teuthology.orchestra.run.smithi150.stdout:1885 read 0x3fad5b4 thru 0x3fb2f95 (0x59e2 bytes) 2024-09-21T23:10:33.074 INFO:teuthology.orchestra.run.smithi150.stdout:1886 read 0x662fc41 thru 0x6631f30 (0x22f0 bytes) 2024-09-21T23:10:33.074 INFO:teuthology.orchestra.run.smithi150.stdout:1889 clone 20 order 20 su 32768 sc 8 2024-09-21T23:10:33.088 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:33.176 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:33.176+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:33.251 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:33.248+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:33.254 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:33.254 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:33.268 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:33.356 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:33.356+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:33.437 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:33.437 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:33.451 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:33.492 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:33.492+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:33.554 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:33.552+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:33.610 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:33.610 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:33.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:33.655 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:33.652+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:33.665 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:10:33.755 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:33.752+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:33.766 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone19 from 0x9ec8805 (overlap 0x1840480) to 0x15d3740 2024-09-21T23:10:33.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:33.795 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:33.795 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:33.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:33.856 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:33.852+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:33.887 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-21T23:10:33.893 INFO:teuthology.orchestra.run.smithi150.stdout:1890 read 0x85b1d7 thru 0x85bec1 (0xceb bytes) 2024-09-21T23:10:33.894 INFO:teuthology.orchestra.run.smithi150.stdout:1891 punch from 0x9a07639 to 0x9a0c0fd, (0x4ac4 bytes) 2024-09-21T23:10:33.900 INFO:teuthology.orchestra.run.smithi150.stdout:1892 read 0x5a2ce9 thru 0x5a323c (0x554 bytes) 2024-09-21T23:10:33.900 INFO:teuthology.orchestra.run.smithi150.stdout:1894 read 0x8af528a thru 0x8b03fc5 (0xed3c bytes) 2024-09-21T23:10:33.902 INFO:teuthology.orchestra.run.smithi150.stdout:1895 read 0x9106329 thru 0x910c132 (0x5e0a bytes) 2024-09-21T23:10:33.903 INFO:teuthology.orchestra.run.smithi150.stdout:1896 write 0x20e9c7 thru 0x21cb38 (0xe172 bytes) 2024-09-21T23:10:33.927 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:33.928 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:33.943 INFO:teuthology.orchestra.run.smithi150.stdout:1897 punch from 0x4da830d to 0x4db5d04, (0xd9f7 bytes) 2024-09-21T23:10:33.944 INFO:teuthology.orchestra.run.smithi150.stdout:1898 write 0x7cd2aff thru 0x7cd8451 (0x5953 bytes) 2024-09-21T23:10:33.946 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:33.946 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:33.956 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:33.956+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:33.964 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:33.965 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:33.979 INFO:teuthology.orchestra.run.smithi150.stdout:1899 trunc from 0x9ec8805 to 0x1784305 2024-09-21T23:10:33.982 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:33.983 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:33.983 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:33.988 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:33.988 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:33.988 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:33.988 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:34.009 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:34.010 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:34.010 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:34.035 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:34.035 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:34.057 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:34.052+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:34.060 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:34.061 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:34.065 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:34.065 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:34.090 INFO:teuthology.orchestra.run.smithi150.stdout:1900 write 0x9ec9759 thru 0x9eca77e (0x1026 bytes) 2024-09-21T23:10:34.093 INFO:teuthology.orchestra.run.smithi150.stdout:1901 write 0x4246777 thru 0x424e633 (0x7ebd bytes) 2024-09-21T23:10:34.100 INFO:teuthology.orchestra.run.smithi150.stdout:1903 write 0x37caa1b thru 0x37ceafc (0x40e2 bytes) 2024-09-21T23:10:34.105 INFO:teuthology.orchestra.run.smithi150.stdout:1904 write 0x1df3582 thru 0x1e01fe8 (0xea67 bytes) 2024-09-21T23:10:34.115 INFO:teuthology.orchestra.run.smithi150.stdout:1905 write 0x8b372c9 thru 0x8b44803 (0xd53b bytes) 2024-09-21T23:10:34.126 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.127 INFO:teuthology.orchestra.run.smithi150.stdout:1908 write 0x45335c7 thru 0x45409f5 (0xd42f bytes) 2024-09-21T23:10:34.135 INFO:teuthology.orchestra.run.smithi150.stdout:1909 read 0x8d55433 thru 0x8d60339 (0xaf07 bytes) 2024-09-21T23:10:34.135 INFO:teuthology.orchestra.run.smithi150.stdout:1910 punch from 0x42eae90 to 0x42edea6, (0x3016 bytes) 2024-09-21T23:10:34.135 INFO:teuthology.orchestra.run.smithi150.stdout:1912 punch from 0x69c9b4e to 0x69ca5a5, (0xa57 bytes) 2024-09-21T23:10:34.135 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:34.136 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:34.136 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:34.136 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:34.136 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:34.146 INFO:teuthology.orchestra.run.smithi150.stdout:1913 writesame 0xd45d6fe thru 0xd466e6c data_size 0x1(0x976f bytes) 2024-09-21T23:10:34.153 INFO:teuthology.orchestra.run.smithi150.stdout:1914 write 0xd75bdc7 thru 0xd75e5b7 (0x27f1 bytes) 2024-09-21T23:10:34.155 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:34.158 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:34.156+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:34.160 INFO:teuthology.orchestra.run.smithi150.stdout:1915 read 0x16f0202 thru 0x16f1fb9 (0x1db8 bytes) 2024-09-21T23:10:34.162 INFO:teuthology.orchestra.run.smithi150.stdout:1918 read 0x88adfe3 thru 0x88b0096 (0x20b4 bytes) 2024-09-21T23:10:34.162 INFO:teuthology.orchestra.run.smithi150.stdout:1919 write 0x8b9d3ab thru 0x8ba0e0f (0x3a65 bytes) 2024-09-21T23:10:34.176 INFO:teuthology.orchestra.run.smithi150.stdout:1920 trunc from 0xd75e5b8 to 0x9c4a6bc 2024-09-21T23:10:34.210 INFO:teuthology.orchestra.run.smithi150.stdout:1921 trunc from 0x9c4a6bc to 0x16eeea8 2024-09-21T23:10:34.241 INFO:teuthology.orchestra.run.smithi150.stdout:1922 writesame 0x1810986 thru 0x1816f55 data_size 0x48(0x65d0 bytes) 2024-09-21T23:10:34.241 INFO:teuthology.orchestra.run.smithi150.stdout:1923 punch from 0xe5cd3 to 0xe7e39, (0x2166 bytes) 2024-09-21T23:10:34.242 INFO:teuthology.orchestra.run.smithi150.stdout:1924 read 0x72909e thru 0x72b489 (0x23ec bytes) 2024-09-21T23:10:34.242 INFO:teuthology.orchestra.run.smithi150.stdout:1927 read 0x481677 thru 0x487582 (0x5f0c bytes) 2024-09-21T23:10:34.243 INFO:teuthology.orchestra.run.smithi150.stdout:1928 trunc from 0x1816f56 to 0x26472fb 2024-09-21T23:10:34.251 INFO:teuthology.orchestra.run.smithi150.stdout:1929 read 0x10e84b2 thru 0x10ed00f (0x4b5e bytes) 2024-09-21T23:10:34.259 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:34.256+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:34.275 INFO:teuthology.orchestra.run.smithi150.stdout:1930 writesame 0xcd27c95 thru 0xcd32f35 data_size 0x3(0xb2a1 bytes) 2024-09-21T23:10:34.275 INFO:teuthology.orchestra.run.smithi150.stdout:1931 punch from 0x1dd8206 to 0x1de43a7, (0xc1a1 bytes) 2024-09-21T23:10:34.275 INFO:teuthology.orchestra.run.smithi150.stdout:1932 read 0xb1d6e90 thru 0xb1e38cb (0xca3c bytes) 2024-09-21T23:10:34.275 INFO:teuthology.orchestra.run.smithi150.stdout:1933 write 0x933e312 thru 0x934243c (0x412b bytes) 2024-09-21T23:10:34.284 INFO:teuthology.orchestra.run.smithi150.stdout:1934 read 0xbee91b8 thru 0xbee96ff (0x548 bytes) 2024-09-21T23:10:34.284 INFO:teuthology.orchestra.run.smithi150.stdout:1936 read 0x3dea24b thru 0x3dedb5e (0x3914 bytes) 2024-09-21T23:10:34.284 INFO:teuthology.orchestra.run.smithi150.stdout:1937 read 0x4777d2 thru 0x477e8b (0x6ba bytes) 2024-09-21T23:10:34.291 INFO:teuthology.orchestra.run.smithi150.stdout:1938 trunc from 0xcd32f36 to 0x7f64611 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4224.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 48.579314144000001, 2024-09-21T23:10:34.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039594890000000001, 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716125+0000", 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716126+0000", 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716131+0000", 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716144+0000", 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2267e-05 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716158+0000", 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4731e-05 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716239+0000", 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1013999999999994e-05 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.716271+0000", 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1535999999999998e-05 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720016+0000", 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037447399999999999 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:09:45.720084+0000", 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8813999999999995e-05 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:2850 3.4 e30/21 3:2298a721:::rbd_data.114912f5d0f.000000000000000b:head v 30'30, mlcod=30'28)", 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.269422721000002, 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033350860000000001, 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026016+0000", 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026017+0000", 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.68e-07 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026023+0000", 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1959999999999996e-06 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026033+0000", 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8120000000000002e-06 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026042+0000", 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7059999999999993e-06 2024-09-21T23:10:34.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026127+0000", 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4973000000000001e-05 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.026145+0000", 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8422000000000001e-05 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029303+0000", 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031578980000000001 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:06.029351+0000", 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8411000000000001e-05 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:4287 3.0 e31/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 31'16, mlcod=31'14)", 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:34.305 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.266070709000001, 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015395872, 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029368+0000", 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.029371+0000", 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2759999999999999e-06 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031303+0000", 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019322359999999999 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031331+0000", 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8408999999999999e-05 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031342+0000", 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1086e-05 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031428+0000", 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5990999999999997e-05 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.031454+0000", 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044695+0000", 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013240784 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.044764+0000", 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9623999999999996e-05 2024-09-21T23:10:34.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6012 3.4 e31/21 3:21794b57:::rbd_data.1149d3a069d9.0000000000000003:head v 31'34, mlcod=30'31)", 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.027047473, 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094147550000000003, 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.310 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268392+0000", 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.268393+0000", 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.72e-06 2024-09-21T23:10:34.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269210+0000", 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00081656700000000003 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269235+0000", 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5000000000000001e-05 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269248+0000", 2024-09-21T23:10:34.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3393e-05 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269314+0000", 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6180000000000007e-05 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.269341+0000", 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6554999999999999e-05 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277723+0000", 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083823960000000003 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.277806+0000", 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2943999999999997e-05 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.314 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:6173 3.0 e32/21 3:065765e7:::rbd_data.1149d3a069d9.0000000000000009:head v 32'22, mlcod=31'19)", 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 26.697005587, 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002753003, 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598433+0000", 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598435+0000", 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8870000000000001e-06 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598445+0000", 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4949999999999997e-06 2024-09-21T23:10:34.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598464+0000", 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8720999999999999e-05 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598473+0000", 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5980000000000003e-06 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598540+0000", 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7096999999999998e-05 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.598564+0000", 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3288000000000001e-05 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601136+0000", 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025726099999999999 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.601186+0000", 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0306999999999998e-05 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11716 3.9 3:94516a46:::rbd_data.1149e13ad964.000000000000006f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.365606246999999, 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0097894799999999997, 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11716 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929833+0000", 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929835+0000", 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2529999999999999e-06 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929846+0000", 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0859000000000001e-05 2024-09-21T23:10:34.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929869+0000", 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3450000000000001e-05 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929879+0000", 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7270000000000002e-06 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.929973+0000", 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3917000000000002e-05 2024-09-21T23:10:34.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.930106+0000", 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013346199999999999 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939479+0000", 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.009372755 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939522+0000", 2024-09-21T23:10:34.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3025999999999998e-05 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:10.939622+0000", 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010003100000000001 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:12080 3.0 e36/21 3:003353e5:::rbd_data.1149e13ad964.000000000000003f:head v 36'38, mlcod=35'35)", 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 21.708351923999999, 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045448859999999997, 2024-09-21T23:10:34.324 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587087+0000", 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587090+0000", 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.367e-06 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587109+0000", 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9816000000000001e-05 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587158+0000", 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8513999999999997e-05 2024-09-21T23:10:34.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587179+0000", 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587395+0000", 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021613099999999999 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.587449+0000", 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3316000000000002e-05 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.591513+0000", 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040644160000000004 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:12.591632+0000", 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011884599999999999 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21566 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.030117725, 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002480384, 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21566 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.329 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265321+0000", 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265324+0000", 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9500000000000001e-06 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265341+0000", 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6303e-05 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265373+0000", 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2427e-05 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265408+0000", 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5126000000000002e-05 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265555+0000", 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014723500000000001 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.265818+0000", 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026269800000000001 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267587+0000", 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017692000000000001 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267679+0000", 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1922999999999994e-05 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.267802+0000", 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012252199999999999 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21574 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.009433484999999, 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024147919999999998, 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21574 2024-09-21T23:10:34.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286006+0000", 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286008+0000", 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286021+0000", 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2714000000000001e-05 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286051+0000", 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0633000000000003e-05 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286082+0000", 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0213e-05 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286304+0000", 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000222112 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.286570+0000", 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026670399999999999 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288119+0000", 2024-09-21T23:10:34.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0015483579999999999 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288208+0000", 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9133000000000002e-05 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.288420+0000", 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021242900000000001 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:21988 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.461160469999999, 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024888240000000002, 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21988 2024-09-21T23:10:34.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834279+0000", 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834281+0000", 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-06 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834294+0000", 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2306e-05 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834323+0000", 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.885e-05 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834337+0000", 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4735999999999999e-05 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834488+0000", 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015039599999999999 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.834745+0000", 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000257385 2024-09-21T23:10:34.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836546+0000", 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018004220000000001 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836638+0000", 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2483000000000002e-05 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.836767+0000", 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012939900000000001 2024-09-21T23:10:34.343 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:22077 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.228022105000001, 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025167950000000001, 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 22077 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067417+0000", 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067420+0000", 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7829999999999999e-06 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067434+0000", 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4116e-05 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067467+0000", 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2790999999999998e-05 2024-09-21T23:10:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067489+0000", 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067656+0000", 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016664100000000001 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.067913+0000", 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000257154 2024-09-21T23:10:34.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069732+0000", 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001819036 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069820+0000", 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7590000000000007e-05 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.069934+0000", 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011401200000000001 2024-09-21T23:10:34.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:22098 3.9 3:93765d1f:::rbd_header.1149c115bf92:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 16.719021193, 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0066791929999999999, 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 22098 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576418+0000", 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576421+0000", 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6090000000000001e-06 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576439+0000", 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7802e-05 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576481+0000", 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1653999999999998e-05 2024-09-21T23:10:34.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576507+0000", 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6001000000000001e-05 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576679+0000", 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017189700000000001 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.576707+0000", 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8608000000000001e-05 2024-09-21T23:10:34.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.580886+0000", 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041785920000000001 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.581057+0000", 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000170915 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.582894+0000", 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018374070000000001 2024-09-21T23:10:34.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.582981+0000", 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6436999999999999e-05 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:17.583097+0000", 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116271 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29041 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.332182113, 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024231119999999998, 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29041 2024-09-21T23:10:34.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963257+0000", 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963260+0000", 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.224e-06 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963272+0000", 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1817999999999999e-05 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963301+0000", 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9268000000000002e-05 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963315+0000", 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3756e-05 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963483+0000", 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167863 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.963752+0000", 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026917 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965465+0000", 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001713259 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965559+0000", 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3521000000000001e-05 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:24.965680+0000", 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121233 2024-09-21T23:10:34.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29116 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.1948649670000009, 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024782770000000001, 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29116 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100574+0000", 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100577+0000", 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100590+0000", 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2959000000000001e-05 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100620+0000", 2024-09-21T23:10:34.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9833e-05 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100634+0000", 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4167e-05 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.100794+0000", 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015995500000000001 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.101056+0000", 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026250900000000002 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.102829+0000", 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001772259 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.102924+0000", 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4874e-05 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.103052+0000", 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000128856 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29244 3.9 3:95c8871e:::rbd_data.11499299167c.0000000000000302:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9888358670000006, 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080285210000000003, 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29244 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306603+0000", 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306606+0000", 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306620+0000", 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3978999999999999e-05 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306642+0000", 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2557000000000001e-05 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306655+0000", 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2894000000000001e-05 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306715+0000", 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9485999999999999e-05 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.306804+0000", 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9603000000000003e-05 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314472+0000", 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076677029999999997 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314552+0000", 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0275e-05 2024-09-21T23:10:34.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:25.314632+0000", 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9599999999999997e-05 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:29590 3.9 3:936975a0:::rbd_header.11499299167c:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.6681698130000004, 2024-09-21T23:10:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058105209999999999, 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 29590 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627269+0000", 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627273+0000", 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7069999999999998e-06 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627288+0000", 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5129e-05 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627322+0000", 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3494999999999997e-05 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627347+0000", 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5191e-05 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627458+0000", 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011141000000000001 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op must wait for map", 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.627467+0000", 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.630921+0000", 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00345467 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.631080+0000", 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015890500000000001 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.632864+0000", 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017839729999999999 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.632961+0000", 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6784e-05 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:26.633080+0000", 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001189 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:33981 3.9 3:9114e073:::rbd_data.114937d450a8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:30.362164+0000", 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.93327487, 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027857300000000002, 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 33981 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362164+0000", 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362164+0000", 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362167+0000", 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4509999999999999e-06 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362527+0000", 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036011300000000001 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362547+0000", 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0239000000000001e-05 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362556+0000", 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4839999999999993e-06 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362630+0000", 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3718000000000001e-05 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.362767+0000", 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013664999999999999 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.364535+0000", 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017677859999999999 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.364711+0000", 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017594400000000001 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.364950+0000", 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023934499999999999 2024-09-21T23:10:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:34172 3.9 3:9114e073:::rbd_data.114937d450a8.0000000000000001:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:30.843121+0000", 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.4523182179999998, 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0026090190000000002, 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 34172 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843121+0000", 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843121+0000", 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843124+0000", 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6630000000000002e-06 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843135+0000", 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.185e-05 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843166+0000", 2024-09-21T23:10:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0985000000000002e-05 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843188+0000", 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1574999999999999e-05 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843363+0000", 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017465100000000001 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.843624+0000", 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026182700000000001 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.845375+0000", 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001751046 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.845524+0000", 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014844899999999999 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:30.845730+0000", 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205973 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:34240 3.9 3:9114e073:::rbd_data.114937d450a8.0000000000000001:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:31.013400+0000", 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.2820385729999999, 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0032463180000000002, 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 34240 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013400+0000", 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013400+0000", 2024-09-21T23:10:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013403+0000", 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5170000000000002e-06 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013411+0000", 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0949999999999996e-06 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013435+0000", 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4141000000000001e-05 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013446+0000", 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0587999999999999e-05 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013620+0000", 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000173981 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.013876+0000", 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000256197 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.016383+0000", 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025070959999999999 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.016514+0000", 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000130646 2024-09-21T23:10:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.016647+0000", 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013305700000000001 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:34354 3.9 3:9114e073:::rbd_data.114937d450a8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:31.289989+0000", 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.0054505040000001, 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054515539999999999, 2024-09-21T23:10:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 34354 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.289989+0000", 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.289989+0000", 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.289991+0000", 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.289998+0000", 2024-09-21T23:10:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2860000000000002e-06 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.290015+0000", 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6636000000000001e-05 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.290023+0000", 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.551e-06 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.290073+0000", 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9895999999999998e-05 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.290125+0000", 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1826999999999997e-05 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.295277+0000", 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.005152172 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.295342+0000", 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5457999999999996e-05 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.295440+0000", 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7712000000000002e-05 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.397 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:34.398 INFO:teuthology.orchestra.run.smithi150.stdout:1939 read 0x28ef681 thru 0x28fdcd2 (0xe652 bytes) 2024-09-21T23:10:34.399 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1942 writesame 0xed1a1fd thru 0xed1aaac data_size 0x8(0x8b0 bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1943 read 0x3e46117 thru 0x3e47b5d (0x1a47 bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1945 write 0xa752357 thru 0xa752d77 (0xa21 bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1946 writesame 0x4d4c624 thru 0x4d581be data_size 0x7(0xbb9b bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1950 write 0x7c5f717 thru 0x7c6baa1 (0xc38b bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1951 write 0x9dc8b55 thru 0x9dc9790 (0xc3c bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1952 punch from 0x185893f to 0x18677f3, (0xeeb4 bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1953 writesame 0x963519e thru 0x963611c data_size 0x1(0xf7f bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1954 writesame 0xb2fbac9 thru 0xb3045cc data_size 0x3e(0x8b04 bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1955 write 0xcd9253e thru 0xcd9d1c5 (0xac88 bytes) 2024-09-21T23:10:34.401 INFO:teuthology.orchestra.run.smithi150.stdout:1956 trunc from 0xed1aaad to 0xc04af09 2024-09-21T23:10:34.402 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:34.396+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:34.419 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.419 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.419 INFO:teuthology.orchestra.run.smithi150.stdout:1959 punch from 0x952c350 to 0x953b56c, (0xf21c bytes) 2024-09-21T23:10:34.419 INFO:teuthology.orchestra.run.smithi150.stdout:1962 punch from 0xbf1a0f5 to 0xbf24a7e, (0xa989 bytes) 2024-09-21T23:10:34.420 INFO:teuthology.orchestra.run.smithi150.stdout:1964 write 0x60a19c9 thru 0x60af832 (0xde6a bytes) 2024-09-21T23:10:34.433 INFO:teuthology.orchestra.run.smithi150.stdout:1965 trunc from 0xc04af09 to 0xd9bc5f8 2024-09-21T23:10:34.442 INFO:teuthology.orchestra.run.smithi150.stdout:1967 write 0x85a2456 thru 0x85b0a53 (0xe5fe bytes) 2024-09-21T23:10:34.453 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.453 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.453 INFO:teuthology.orchestra.run.smithi150.stdout:1970 read 0x5556ec4 thru 0x55617b1 (0xa8ee bytes) 2024-09-21T23:10:34.453 INFO:teuthology.orchestra.run.smithi150.stdout:1971 writesame 0x8a55dd9 thru 0x8a5c5d0 data_size 0xc(0x67f8 bytes) 2024-09-21T23:10:34.461 INFO:teuthology.orchestra.run.smithi150.stdout:1972 write 0x870d3b9 thru 0x870edd9 (0x1a21 bytes) 2024-09-21T23:10:34.467 INFO:teuthology.orchestra.run.smithi150.stdout:1973 read 0xc6bf84a thru 0xc6c2922 (0x30d9 bytes) 2024-09-21T23:10:34.467 INFO:teuthology.orchestra.run.smithi150.stdout:1974 read 0x9f8b575 thru 0x9f8d839 (0x22c5 bytes) 2024-09-21T23:10:34.468 INFO:teuthology.orchestra.run.smithi150.stdout:1975 trunc from 0xd9bc5f8 to 0xe03c994 2024-09-21T23:10:34.477 INFO:teuthology.orchestra.run.smithi150.stdout:1976 write 0x14f5cfa thru 0x1502a88 (0xcd8f bytes) 2024-09-21T23:10:34.480 INFO:teuthology.orchestra.run.smithi150.stdout:1978 punch from 0x8d369ae to 0x8d4578c, (0xedde bytes) 2024-09-21T23:10:34.480 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.480 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.480 INFO:teuthology.orchestra.run.smithi150.stdout:1981 write 0x7e48918 thru 0x7e50ea0 (0x8589 bytes) 2024-09-21T23:10:34.490 INFO:teuthology.orchestra.run.smithi150.stdout:1982 punch from 0x78226d6 to 0x7826eb8, (0x47e2 bytes) 2024-09-21T23:10:34.490 INFO:teuthology.orchestra.run.smithi150.stdout:1986 read 0xa0be17d thru 0xa0cc2cf (0xe153 bytes) 2024-09-21T23:10:34.490 INFO:teuthology.orchestra.run.smithi150.stdout:1988 write 0x53fa5fe thru 0x53fe8c2 (0x42c5 bytes) 2024-09-21T23:10:34.500 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:34.496+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:34.507 INFO:teuthology.orchestra.run.smithi150.stdout:1993 trunc from 0xe03c994 to 0x85bb89d 2024-09-21T23:10:34.537 INFO:teuthology.orchestra.run.smithi150.stdout:1994 writesame 0x8a2c59e thru 0x8a2e97f data_size 0x6(0x23e2 bytes) 2024-09-21T23:10:34.546 INFO:teuthology.orchestra.run.smithi150.stdout:1995 trunc from 0x8a2e980 to 0x2e8c176 2024-09-21T23:10:34.576 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:34.576 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:34.576 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:34.598 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:34.599 INFO:teuthology.orchestra.run.smithi150.stdout:1996 write 0xa6c5a2f thru 0xa6d46f7 (0xecc9 bytes) 2024-09-21T23:10:34.609 INFO:teuthology.orchestra.run.smithi150.stdout:1997 write 0x74231fe thru 0x74282ab (0x50ae bytes) 2024-09-21T23:10:34.610 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:34.608+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:34.630 INFO:teuthology.orchestra.run.smithi150.stdout:1998 write 0xddc423c thru 0xddc7cc9 (0x3a8e bytes) 2024-09-21T23:10:34.637 INFO:teuthology.orchestra.run.smithi150.stdout:1999 punch from 0x5c1f3a8 to 0x5c2d3cf, (0xe027 bytes) 2024-09-21T23:10:34.639 INFO:teuthology.orchestra.run.smithi150.stdout:2000 trunc from 0xddc7cca to 0xc1600e2 2024-09-21T23:10:34.655 INFO:teuthology.orchestra.run.smithi150.stdout:2002 writesame 0x648c396 thru 0x6490454 data_size 0x5(0x40bf bytes) 2024-09-21T23:10:34.665 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.665 INFO:teuthology.orchestra.run.smithi150.stdout:2010 punch from 0x4207508 to 0x42146d3, (0xd1cb bytes) 2024-09-21T23:10:34.665 INFO:teuthology.orchestra.run.smithi150.stdout:2012 writesame 0x5e230b2 thru 0x5e2c024 data_size 0x1(0x8f73 bytes) 2024-09-21T23:10:34.688 INFO:teuthology.orchestra.run.smithi150.stdout:2013 write 0x415db63 thru 0x416bf26 (0xe3c4 bytes) 2024-09-21T23:10:34.699 INFO:teuthology.orchestra.run.smithi150.stdout:2014 writesame 0x95a1bbc thru 0x95a7065 data_size 0x1(0x54aa bytes) 2024-09-21T23:10:34.700 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:34.700+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:34.732 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.732 INFO:teuthology.orchestra.run.smithi150.stdout:2016 read 0x41b3ea8 thru 0x41c051f (0xc678 bytes) 2024-09-21T23:10:34.732 INFO:teuthology.orchestra.run.smithi150.stdout:2018 read 0x1ca3b01 thru 0x1ca4b57 (0x1057 bytes) 2024-09-21T23:10:34.732 INFO:teuthology.orchestra.run.smithi150.stdout:2019 writesame 0xa495949 thru 0xa4a0d45 data_size 0x1(0xb3fd bytes) 2024-09-21T23:10:34.740 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:34.740 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:34.756 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:34.778 INFO:teuthology.orchestra.run.smithi150.stdout:2020 read 0xb4ada10 thru 0xb4bb0cf (0xd6c0 bytes) 2024-09-21T23:10:34.778 INFO:teuthology.orchestra.run.smithi150.stdout:2022 write 0x9e89fe9 thru 0x9e925ad (0x85c5 bytes) 2024-09-21T23:10:34.786 INFO:teuthology.orchestra.run.smithi150.stdout:2023 punch from 0x925a539 to 0x925fd72, (0x5839 bytes) 2024-09-21T23:10:34.801 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:34.800+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:34.802 INFO:teuthology.orchestra.run.smithi150.stdout:2024 trunc from 0xc1600e2 to 0x10d00ab 2024-09-21T23:10:34.824 INFO:teuthology.orchestra.run.smithi150.stdout:2027 punch from 0x13ca88 to 0x148e2e, (0xc3a6 bytes) 2024-09-21T23:10:34.848 INFO:teuthology.orchestra.run.smithi150.stdout:2029 writesame 0xb0217bf thru 0xb03150d data_size 0x1(0xfd4f bytes) 2024-09-21T23:10:34.848 INFO:teuthology.orchestra.run.smithi150.stdout:2030 read 0x1ec466f thru 0x1ed0427 (0xbdb9 bytes) 2024-09-21T23:10:34.848 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.848 INFO:teuthology.orchestra.run.smithi150.stdout:2032 read 0x7f9ac1f thru 0x7fa9b61 (0xef43 bytes) 2024-09-21T23:10:34.848 INFO:teuthology.orchestra.run.smithi150.stdout:2033 flatten 2024-09-21T23:10:34.902 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:34.900+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:34.927 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:34.927 INFO:teuthology.orchestra.run.smithi150.stdout:2035 read 0x301173d thru 0x301d6ad (0xbf71 bytes) 2024-09-21T23:10:34.927 INFO:teuthology.orchestra.run.smithi150.stdout:2036 writesame 0x599e4f0 thru 0x59a4c1d data_size 0x2f(0x672e bytes) 2024-09-21T23:10:34.928 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4902 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.928 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.748669031999999, 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030793945, 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4902 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.37e-07 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179183+0000", 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9280000000000002e-06 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179192+0000", 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7020000000000007e-06 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179198+0000", 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6780000000000003e-06 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201407+0000", 2024-09-21T23:10:34.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022208301999999999 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201500+0000", 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3231000000000002e-05 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.209974+0000", 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084735669999999996 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.932 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4903 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2359296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.748641057, 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031240032000000001, 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4903 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:34.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179211+0000", 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6630000000000002e-06 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179220+0000", 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8480000000000007e-06 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179226+0000", 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2650000000000002e-06 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210016+0000", 2024-09-21T23:10:34.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030790128 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210066+0000", 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9713000000000003e-05 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210448+0000", 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038190499999999998 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.936 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4904 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.74861383, 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031636297000000001, 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4904 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:34.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179238+0000", 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5349999999999999e-06 2024-09-21T23:10:34.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179247+0000", 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6449999999999994e-06 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179260+0000", 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3431999999999999e-05 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210480+0000", 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031220121 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210550+0000", 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0352000000000007e-05 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210871+0000", 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000320701 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4905 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.748578665, 2024-09-21T23:10:34.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036964023999999998, 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4905 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179271+0000", 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179274+0000", 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1350000000000001e-06 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179283+0000", 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1779999999999992e-06 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179290+0000", 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210903+0000", 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031613162 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210976+0000", 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2591999999999999e-05 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216234+0000", 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052583309999999998 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4906 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.748548922000001, 2024-09-21T23:10:34.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037394366999999998, 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4906 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179303+0000", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8219999999999998e-06 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179312+0000", 2024-09-21T23:10:34.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6419999999999996e-06 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179319+0000", 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6449999999999999e-06 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216272+0000", 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036953142000000001 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216352+0000", 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0044999999999997e-05 2024-09-21T23:10:34.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216694+0000", 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034238399999999999 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5226 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.735152524, 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029942795000000001, 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5226 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192697+0000", 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7199999999999998e-07 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192701+0000", 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8e-06 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192712+0000", 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1156e-05 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192724+0000", 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2551999999999999e-05 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222110+0000", 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029385620000000001 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222191+0000", 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1520000000000006e-05 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222639+0000", 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044747500000000002 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5227 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.735113183999999, 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030372671, 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.952 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5227 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192736+0000", 2024-09-21T23:10:34.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192742+0000", 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2260000000000002e-06 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192753+0000", 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0762e-05 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192761+0000", 2024-09-21T23:10:34.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.14e-06 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222674+0000", 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029912979999999999 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222754+0000", 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9816999999999998e-05 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223108+0000", 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035405799999999998 2024-09-21T23:10:34.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5228 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.73507627, 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0320895, 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5228 2024-09-21T23:10:34.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192773+0000", 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:10:34.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192776+0000", 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2380000000000002e-06 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192787+0000", 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0764999999999999e-05 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192795+0000", 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3000000000000002e-06 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223154+0000", 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030358672999999999 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223230+0000", 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6268999999999998e-05 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224862+0000", 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001631635 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5229 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.735042087, 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032496191000000001, 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5229 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5199999999999996e-07 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192810+0000", 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192820+0000", 2024-09-21T23:10:34.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0132000000000001e-05 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192829+0000", 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5900000000000008e-06 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224912+0000", 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032083410999999999 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224985+0000", 2024-09-21T23:10:34.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2455999999999999e-05 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225303+0000", 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031788999999999998 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5230 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.735007413000002, 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033015543000000001, 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5230 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:34.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192842+0000", 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192845+0000", 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192856+0000", 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1008e-05 2024-09-21T23:10:34.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192865+0000", 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225398+0000", 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032533617000000001 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225489+0000", 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1038999999999994e-05 2024-09-21T23:10:34.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225857+0000", 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000367597 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5231 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.734973491000002, 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033349594000000003, 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.968 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5231 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:34.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192876+0000", 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192879+0000", 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2059999999999999e-06 2024-09-21T23:10:34.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192890+0000", 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0702e-05 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192898+0000", 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3299999999999999e-06 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225903+0000", 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033004622999999997 2024-09-21T23:10:34.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225981+0000", 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7990000000000004e-05 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226225+0000", 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024411499999999999 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5232 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3801088~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:34.972 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.734939521000001, 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033795809000000003, 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5232 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192910+0000", 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1800000000000005e-07 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192913+0000", 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1669999999999999e-06 2024-09-21T23:10:34.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192924+0000", 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0692e-05 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192936+0000", 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2414e-05 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226252+0000", 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03331626 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226309+0000", 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6733e-05 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226705+0000", 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00039582499999999999 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5406 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.729426273000001, 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029727338999999998, 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5406 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198423+0000", 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7100000000000001e-07 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198427+0000", 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198436+0000", 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198446+0000", 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6360000000000006e-06 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227686+0000", 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029240289999999999 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227764+0000", 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7676000000000006e-05 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228150+0000", 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038574300000000002 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5407 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:34.980 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.729392395000001, 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030154378999999999, 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5407 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198457+0000", 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198460+0000", 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3409999999999999e-06 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198470+0000", 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198478+0000", 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7130000000000005e-06 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228216+0000", 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029738243000000001 2024-09-21T23:10:34.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228290+0000", 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3461999999999996e-05 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228611+0000", 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032108100000000002 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.984 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5408 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.729360928999998, 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030920828000000001, 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5408 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:34.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198491+0000", 2024-09-21T23:10:34.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198501+0000", 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5719999999999992e-06 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198508+0000", 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2470000000000002e-06 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228720+0000", 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030211598999999999 2024-09-21T23:10:34.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228851+0000", 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013098299999999999 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229409+0000", 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055781799999999996 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5412 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.727255920000001, 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030113958, 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5412 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4099999999999998e-07 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200626+0000", 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2672999999999997e-05 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200637+0000", 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1248e-05 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200646+0000", 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6559999999999998e-06 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230319+0000", 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029672765 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230378+0000", 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8696999999999997e-05 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230707+0000", 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032917800000000001 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.853162676, 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:34.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:34.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:34.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:34.998 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.687395824999999, 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:34.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:35.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:35.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:21897 3.5 e37/21 3:a0a2d2ee:::rbd_data.1149c115bf92.0000000000000000:head v 37'78, mlcod=37'76)", 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.435672652000001, 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037373174000000002, 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:35.004 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492178+0000", 2024-09-21T23:10:35.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9400000000000001e-06 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499469+0000", 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072913370000000002 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499506+0000", 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6412999999999998e-05 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499524+0000", 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8097e-05 2024-09-21T23:10:35.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499683+0000", 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000159631 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499726+0000", 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2716999999999998e-05 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.529473+0000", 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029746356000000002 2024-09-21T23:10:35.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.529549+0000", 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6682999999999997e-05 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:27646 3.b 3:d396a686:::rbd_data.11499b8b109.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.572128834000001, 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032669003000000002, 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:35.008 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 27646 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:35.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355723+0000", 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.360977+0000", 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052540290000000003 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361008+0000", 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0840000000000003e-05 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361028+0000", 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9771000000000001e-05 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361182+0000", 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015453799999999999 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361827+0000", 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064513600000000004 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.363660+0000", 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001833131 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388259+0000", 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024598811000000002 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:35.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388305+0000", 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5479000000000003e-05 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388341+0000", 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6009e-05 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388389+0000", 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8229000000000002e-05 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:35.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:35.014 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:35.014 INFO:teuthology.orchestra.run.smithi150.stdout:2038 read 0x4d39229 thru 0x4d4557c (0xc354 bytes) 2024-09-21T23:10:35.015 INFO:teuthology.orchestra.run.smithi150.stdout:2039 write 0x578d8d9 thru 0x579c357 (0xea7f bytes) 2024-09-21T23:10:35.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:35.016 INFO:teuthology.orchestra.run.smithi150.stdout:2040 trunc from 0xb03150e to 0xd1827f0 2024-09-21T23:10:35.016 INFO:teuthology.orchestra.run.smithi150.stdout:2041 writesame 0xc5b1070 thru 0xc5b4ccc data_size 0x1(0x3c5d bytes) 2024-09-21T23:10:35.017 INFO:teuthology.orchestra.run.smithi150.stdout:2042 read 0xa4312ef thru 0xa43bfff (0xad11 bytes) 2024-09-21T23:10:35.017 INFO:teuthology.orchestra.run.smithi150.stdout:2044 read 0x1a0dbbe thru 0x1a12d01 (0x5144 bytes) 2024-09-21T23:10:35.017 INFO:teuthology.orchestra.run.smithi150.stdout:2045 read 0x578581f thru 0x5790a98 (0xb27a bytes) 2024-09-21T23:10:35.017 INFO:teuthology.orchestra.run.smithi150.stdout:2046 read 0x5b6c7f thru 0x5c07f9 (0x9b7b bytes) 2024-09-21T23:10:35.017 INFO:teuthology.orchestra.run.smithi150.stdout:2047 trunc from 0xd1827f0 to 0xd9d137b 2024-09-21T23:10:35.017 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.017 INFO:teuthology.orchestra.run.smithi150.stdout:2049 trunc from 0xd9d137b to 0x6d4d54e 2024-09-21T23:10:35.017 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:35.016+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:35.027 INFO:teuthology.orchestra.run.smithi150.stdout:2050 punch from 0x3ea5ee8 to 0x3eb3ef4, (0xe00c bytes) 2024-09-21T23:10:35.027 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.027 INFO:teuthology.orchestra.run.smithi150.stdout:2053 write 0x1aec29b thru 0x1af7392 (0xb0f8 bytes) 2024-09-21T23:10:35.032 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T23:10:35.032 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:10:35.033 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T23:10:35.033 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-21T23:10:35.037 INFO:teuthology.orchestra.run.smithi150.stdout:2054 writesame 0x33fe670 thru 0x34051a5 data_size 0x1(0x6b36 bytes) 2024-09-21T23:10:35.077 INFO:teuthology.orchestra.run.smithi150.stdout:2055 writesame 0xaafe811 thru 0xaaffb95 data_size 0x1(0x1385 bytes) 2024-09-21T23:10:35.077 INFO:teuthology.orchestra.run.smithi150.stdout:2056 punch from 0x2805aad to 0x280e1a2, (0x86f5 bytes) 2024-09-21T23:10:35.077 INFO:teuthology.orchestra.run.smithi150.stdout:2058 writesame 0xa21bb42 thru 0xa21d260 data_size 0x1(0x171f bytes) 2024-09-21T23:10:35.093 INFO:teuthology.orchestra.run.smithi150.stdout:2059 trunc from 0xaaffb96 to 0xe3b3395 2024-09-21T23:10:35.101 INFO:teuthology.orchestra.run.smithi150.stdout:2060 write 0x63431d4 thru 0x6347c13 (0x4a40 bytes) 2024-09-21T23:10:35.107 INFO:teuthology.orchestra.run.smithi150.stdout:2061 read 0x92ec00f thru 0x92f45ac (0x859e bytes) 2024-09-21T23:10:35.107 INFO:teuthology.orchestra.run.smithi150.stdout:2062 write 0x2e97628 thru 0x2ea2ce3 (0xb6bc bytes) 2024-09-21T23:10:35.116 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:35.116+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:35.117 INFO:teuthology.orchestra.run.smithi150.stdout:2063 write 0xcb4fca0 thru 0xcb5f020 (0xf381 bytes) 2024-09-21T23:10:35.128 INFO:teuthology.orchestra.run.smithi150.stdout:2064 write 0x692d236 thru 0x692d2ac (0x77 bytes) 2024-09-21T23:10:35.133 INFO:teuthology.orchestra.run.smithi150.stdout:2065 punch from 0x66bfec1 to 0x66cfd05, (0xfe44 bytes) 2024-09-21T23:10:35.134 INFO:teuthology.orchestra.run.smithi150.stdout:2066 read 0x703aedd thru 0x70401ab (0x52cf bytes) 2024-09-21T23:10:35.134 INFO:teuthology.orchestra.run.smithi150.stdout:2067 read 0xa7a0da2 thru 0xa7ac231 (0xb490 bytes) 2024-09-21T23:10:35.134 INFO:teuthology.orchestra.run.smithi150.stdout:2073 write 0x40615fb thru 0x406f5d5 (0xdfdb bytes) 2024-09-21T23:10:35.142 INFO:teuthology.orchestra.run.smithi150.stdout:2074 punch from 0xd22b44 to 0xd314d3, (0xe98f bytes) 2024-09-21T23:10:35.143 INFO:teuthology.orchestra.run.smithi150.stdout:2076 write 0x7fadcbe thru 0x7faea97 (0xdda bytes) 2024-09-21T23:10:35.148 INFO:teuthology.orchestra.run.smithi150.stdout:2078 write 0xc48fd6d thru 0xc4941ad (0x4441 bytes) 2024-09-21T23:10:35.153 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:36566 3.5 3:a56418a7:::rbd_object_map.1149a17d43fd:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:35.151822+0000", 2024-09-21T23:10:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.003289251, 2024-09-21T23:10:35.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0033351909999999999, 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 36566 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151822+0000", 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151822+0000", 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151824+0000", 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.03e-06 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151847+0000", 2024-09-21T23:10:35.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2424000000000001e-05 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151875+0000", 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7934999999999999e-05 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151890+0000", 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5288999999999999e-05 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.152027+0000", 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000137243 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.152182+0000", 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000154475 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.152720+0000", 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00053877899999999997 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.155015+0000", 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022941569999999998 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.155147+0000", 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013266999999999999 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:36567 3.5 3:a56418a7:::rbd_object_map.1149a17d43fd:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:35.151941+0000", 2024-09-21T23:10:35.158 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00317038, 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034107819999999998, 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 36567 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151941+0000", 2024-09-21T23:10:35.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151941+0000", 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151942+0000", 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.026e-06 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151955+0000", 2024-09-21T23:10:35.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2733e-05 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151985+0000", 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0354000000000001e-05 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.151995+0000", 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8530000000000004e-06 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.153526+0000", 2024-09-21T23:10:35.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015314 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.153662+0000", 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013524699999999999 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3", 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.154218+0000", 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00055623499999999998 2024-09-21T23:10:35.162 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.4425.0:36566 3.5 e49/21 ondisk, result = 0)", 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:35.154654+0000", 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.000457489, 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000812282, 2024-09-21T23:10:35.163 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.154654+0000", 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.154654+0000", 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.154655+0000", 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.19e-06 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.154663+0000", 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7260000000000002e-06 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.154697+0000", 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4034e-05 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.154710+0000", 2024-09-21T23:10:35.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2859e-05 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.155105+0000", 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039546699999999999 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.155130+0000", 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4972000000000001e-05 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:35.155472+0000", 2024-09-21T23:10:35.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034231700000000002 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 3 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi150.stdout:2079 read 0xb482b6 thru 0xb57258 (0xefa3 bytes) 2024-09-21T23:10:35.167 INFO:teuthology.orchestra.run.smithi150.stdout:2080 write 0xd667a7c thru 0xd6708e0 (0x8e65 bytes) 2024-09-21T23:10:35.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:35.170 INFO:teuthology.orchestra.run.smithi150.stdout:2081 writesame 0x9af1028 thru 0x9af7101 data_size 0x31(0x60da bytes) 2024-09-21T23:10:35.173 INFO:teuthology.orchestra.run.smithi150.stdout:2082 writesame 0x2a3f47f thru 0x2a4da41 data_size 0x1(0xe5c3 bytes) 2024-09-21T23:10:35.194 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.194 INFO:teuthology.orchestra.run.smithi150.stdout:2084 punch from 0x18fdeb9 to 0x18fdf21, (0x68 bytes) 2024-09-21T23:10:35.194 INFO:teuthology.orchestra.run.smithi150.stdout:2085 write 0xd1d5d2d thru 0xd1dea14 (0x8ce8 bytes) 2024-09-21T23:10:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:2087 read 0xdd78087 thru 0xdd81d75 (0x9cef bytes) 2024-09-21T23:10:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:2090 read 0x3688970 thru 0x368fb6c (0x71fd bytes) 2024-09-21T23:10:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:2094 read 0xe11eb3 thru 0xe1cf06 (0xb054 bytes) 2024-09-21T23:10:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:2096 punch from 0xbfcfff4 to 0xbfd9324, (0x9330 bytes) 2024-09-21T23:10:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:2097 writesame 0x71172e2 thru 0x7123a94 data_size 0x1(0xc7b3 bytes) 2024-09-21T23:10:35.217 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:35.216+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:35.232 INFO:teuthology.orchestra.run.smithi150.stdout:2098 trunc from 0xe3b3395 to 0x4ef955c 2024-09-21T23:10:35.248 INFO:teuthology.orchestra.run.smithi150.stdout:2099 read 0x8e6590 thru 0x8e91c3 (0x2c34 bytes) 2024-09-21T23:10:35.255 INFO:teuthology.orchestra.run.smithi150.stdout:2100 trunc from 0x4ef955c to 0x71af8b 2024-09-21T23:10:35.273 INFO:teuthology.orchestra.run.smithi150.stdout:2101 read 0x6aa7 thru 0xa223 (0x377d bytes) 2024-09-21T23:10:35.273 INFO:teuthology.orchestra.run.smithi150.stdout:2103 punch from 0x2e39ed to 0x2f16d4, (0xdce7 bytes) 2024-09-21T23:10:35.297 INFO:teuthology.orchestra.run.smithi150.stdout:2104 write 0xae24b85 thru 0xae326c7 (0xdb43 bytes) 2024-09-21T23:10:35.301 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:35.301 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:35.301 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:35.302 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:35.302 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:35.307 INFO:teuthology.orchestra.run.smithi150.stdout:2105 write 0x5f10b1c thru 0x5f13801 (0x2ce6 bytes) 2024-09-21T23:10:35.318 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:35.316+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:35.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:35.320 INFO:teuthology.orchestra.run.smithi150.stdout:2106 trunc from 0xae326c8 to 0x684a8b9 2024-09-21T23:10:35.344 INFO:teuthology.orchestra.run.smithi150.stdout:2107 trunc from 0x684a8b9 to 0xd16955c 2024-09-21T23:10:35.350 INFO:teuthology.orchestra.run.smithi150.stdout:2109 writesame 0xc44e7c thru 0xc47e99 data_size 0x3(0x301e bytes) 2024-09-21T23:10:35.355 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.359 INFO:teuthology.orchestra.run.smithi150.stdout:2111 trunc from 0xd16955c to 0xaccb13c 2024-09-21T23:10:35.368 INFO:teuthology.orchestra.run.smithi150.stdout:2115 write 0x3d77b02 thru 0x3d825eb (0xaaea bytes) 2024-09-21T23:10:35.374 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.374 INFO:teuthology.orchestra.run.smithi150.stdout:2117 writesame 0x20895c0 thru 0x209878f data_size 0x92(0xf1d0 bytes) 2024-09-21T23:10:35.380 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.380 INFO:teuthology.orchestra.run.smithi150.stdout:2119 punch from 0x8c34012 to 0x8c37dbf, (0x3dad bytes) 2024-09-21T23:10:35.380 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.380 INFO:teuthology.orchestra.run.smithi150.stdout:2122 read 0x5d32fa1 thru 0x5d3eb6f (0xbbcf bytes) 2024-09-21T23:10:35.380 INFO:teuthology.orchestra.run.smithi150.stdout:2124 write 0x1976276 thru 0x1983c7d (0xda08 bytes) 2024-09-21T23:10:35.387 INFO:teuthology.orchestra.run.smithi150.stdout:2126 writesame 0xa38bb6a thru 0xa38c605 data_size 0xe(0xa9c bytes) 2024-09-21T23:10:35.390 INFO:tasks.ceph.mon.a.smithi150.stderr:2024-09-21T23:10:35.388+0000 7f4d03887640 -1 mon.a@0(leader).osd e49 definitely_dead 0 2024-09-21T23:10:35.407 INFO:teuthology.orchestra.run.smithi150.stdout:2127 trunc from 0xaccb13c to 0x247dd9c 2024-09-21T23:10:35.418 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:35.416+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:35.423 INFO:teuthology.orchestra.run.smithi150.stdout:2128 read 0x1eeeead thru 0x1eef60e (0x762 bytes) 2024-09-21T23:10:35.434 INFO:teuthology.orchestra.run.smithi150.stdout:2129 trunc from 0x247dd9c to 0x805e105 2024-09-21T23:10:35.440 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:35.440 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5124 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.254695775999998, 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046086192999999998, 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.441 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5124 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186881+0000", 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.133e-06 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186887+0000", 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7939999999999999e-06 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186904+0000", 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7193e-05 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186918+0000", 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223027+0000", 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036108460000000002 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223201+0000", 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017483900000000001 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.232966+0000", 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097649039999999996 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5279 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253417135999999, 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045406334999999999, 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.445 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5279 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188160+0000", 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188167+0000", 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0700000000000001e-06 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188188+0000", 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-05 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188205+0000", 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7444000000000001e-05 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233051+0000", 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044846496999999999 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233167+0000", 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115307 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233565+0000", 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039829399999999999 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5280 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:35.449 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253347940000001, 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047857909999999997, 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5280 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188229+0000", 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.237e-06 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188235+0000", 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188254+0000", 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9015999999999999e-05 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188268+0000", 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.395e-05 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233608+0000", 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045339957 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233722+0000", 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011403700000000001 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236086+0000", 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00236359 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5281 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.25328644, 2024-09-21T23:10:35.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048399091999999998, 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5281 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188291+0000", 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188298+0000", 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188315+0000", 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7819999999999999e-05 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188329+0000", 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3383e-05 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236130+0000", 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047800827999999997 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236240+0000", 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110745 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236689+0000", 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044823600000000002 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5282 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.457 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253228561, 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048602487, 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5282 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:35.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188348+0000", 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188355+0000", 2024-09-21T23:10:35.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3029999999999996e-06 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188373+0000", 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8056000000000001e-05 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188386+0000", 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3536999999999999e-05 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236748+0000", 2024-09-21T23:10:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048361617000000003 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236840+0000", 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2075000000000002e-05 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236950+0000", 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109755 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5283 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253085301999999, 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048844519000000003, 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5283 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188493+0000", 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3800000000000001e-06 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188503+0000", 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188526+0000", 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.304e-05 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188543+0000", 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7557000000000001e-05 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237020+0000", 2024-09-21T23:10:35.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048476656 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237187+0000", 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016670199999999999 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237335+0000", 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014857100000000001 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.465 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5284 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253003436, 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049117881000000002, 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5284 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188574+0000", 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188580+0000", 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188598+0000", 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8304e-05 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188611+0000", 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3329000000000001e-05 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237402+0000", 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048790838000000003 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237549+0000", 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146897 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237690+0000", 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141383 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5285 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.25294615, 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049392706000000001, 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5285 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.470 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188631+0000", 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0400000000000005e-07 2024-09-21T23:10:35.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188636+0000", 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8329999999999999e-06 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188655+0000", 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8246000000000001e-05 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188670+0000", 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5440000000000001e-05 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237748+0000", 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049077975000000003 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237890+0000", 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141953 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238022+0000", 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013235499999999999 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5286 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252887047000002, 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049655168, 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5286 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.474 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188690+0000", 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1769999999999999e-06 2024-09-21T23:10:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188696+0000", 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5269999999999998e-06 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188713+0000", 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7836e-05 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188726+0000", 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:10:35.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238077+0000", 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049350964999999997 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238228+0000", 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015027000000000001 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238344+0000", 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116403 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5287 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252117459000001, 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049431115999999997, 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5287 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189460+0000", 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189470+0000", 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189491+0000", 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0743000000000001e-05 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189503+0000", 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2787e-05 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238395+0000", 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048891284 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238519+0000", 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124745 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238890+0000", 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037020400000000001 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5465 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.482 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.244321466999999, 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045516793999999999, 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5465 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197256+0000", 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197262+0000", 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197280+0000", 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197294+0000", 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3983e-05 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242417+0000", 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045122664999999999 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242550+0000", 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013329499999999999 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242771+0000", 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022120399999999999 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5466 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.486 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.244263409999999, 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045852049999999998, 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5466 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197313+0000", 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197321+0000", 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197338+0000", 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7824000000000001e-05 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197353+0000", 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.456e-05 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242853+0000", 2024-09-21T23:10:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045500119999999998 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243010+0000", 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015682599999999999 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243165+0000", 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015463400000000001 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5467 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.244204166999999, 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046123232, 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5467 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.491 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197373+0000", 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197378+0000", 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197397+0000", 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8793000000000001e-05 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197411+0000", 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3726999999999999e-05 2024-09-21T23:10:35.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243249+0000", 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045838320000000002 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243376+0000", 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012760199999999999 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243495+0000", 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011848100000000001 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5639 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.241392556000001, 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045343482999999997, 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5639 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200184+0000", 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200190+0000", 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200209+0000", 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8563000000000002e-05 2024-09-21T23:10:35.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200223+0000", 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4173e-05 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.244967+0000", 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044743567999999997 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245103+0000", 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013634300000000001 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245527+0000", 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042370199999999998 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5640 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.241296698999999, 2024-09-21T23:10:35.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045524901, 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5640 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200280+0000", 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200287+0000", 2024-09-21T23:10:35.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200307+0000", 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0129999999999999e-05 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200321+0000", 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245576+0000", 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045255521 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245701+0000", 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012480100000000001 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245804+0000", 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102933 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5641 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.24123406, 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045750444000000001, 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5641 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200343+0000", 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200351+0000", 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9160000000000001e-06 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200369+0000", 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8490000000000001e-05 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200383+0000", 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3234000000000001e-05 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245850+0000", 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045467883000000001 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245979+0000", 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128641 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246092+0000", 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113143 2024-09-21T23:10:35.507 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5642 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.241173957000001, 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045975347999999999, 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.508 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5642 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200403+0000", 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200409+0000", 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6570000000000002e-06 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200427+0000", 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8306000000000001e-05 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200441+0000", 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4042999999999999e-05 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246144+0000", 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045702495000000003 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246277+0000", 2024-09-21T23:10:35.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133831 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246377+0000", 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9835999999999998e-05 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5643 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.241116430999998, 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046409156, 2024-09-21T23:10:35.512 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5643 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200460+0000", 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200466+0000", 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200483+0000", 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7776999999999999e-05 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200500+0000", 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6852999999999999e-05 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246582+0000", 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046081471999999998 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246725+0000", 2024-09-21T23:10:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143184 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246869+0000", 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014361699999999999 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5644 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.241056358000002, 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046601900000000002, 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.516 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5644 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:35.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200521+0000", 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200526+0000", 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-06 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200544+0000", 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7410000000000001e-05 2024-09-21T23:10:35.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200559+0000", 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5435999999999999e-05 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246920+0000", 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046360902000000002 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247055+0000", 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468200000000001 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247121+0000", 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6712999999999996e-05 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5645 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.240997631999999, 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.04674565, 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:35.520 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5645 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:35.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200579+0000", 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2270000000000001e-06 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200585+0000", 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200605+0000", 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9675999999999999e-05 2024-09-21T23:10:35.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200618+0000", 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509999999999999e-05 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247148+0000", 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046529451999999999 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247219+0000", 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0833000000000004e-05 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:35.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247324+0000", 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105122 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:35.524 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:35.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:35.527 INFO:teuthology.orchestra.run.smithi150.stdout:2130 read 0x2a198ee thru 0x2a227fd (0x8f10 bytes) 2024-09-21T23:10:35.527 INFO:teuthology.orchestra.run.smithi150.stdout:2131 write 0x3d1238 thru 0x3d4f99 (0x3d62 bytes) 2024-09-21T23:10:35.527 INFO:teuthology.orchestra.run.smithi150.stdout:2132 read 0x19cfe14 thru 0x19dcf76 (0xd163 bytes) 2024-09-21T23:10:35.527 INFO:teuthology.orchestra.run.smithi150.stdout:2134 punch from 0x7280bff to 0x7282822, (0x1c23 bytes) 2024-09-21T23:10:35.527 INFO:teuthology.orchestra.run.smithi150.stdout:2135 write 0xaa34276 thru 0xaa43c58 (0xf9e3 bytes) 2024-09-21T23:10:35.527 INFO:teuthology.orchestra.run.smithi150.stdout:2136 write 0x6274bfa thru 0x6278d1e (0x4125 bytes) 2024-09-21T23:10:35.527 INFO:teuthology.orchestra.run.smithi150.stdout:2137 read 0xa353c09 thru 0xa3560e6 (0x24de bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2138 writesame 0x7ca13ee thru 0x7cafcbd data_size 0x95(0xe8d0 bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2140 write 0x8d2a862 thru 0x8d37b39 (0xd2d8 bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2141 read 0x33f0377 thru 0x33f9444 (0x90ce bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2144 write 0x3e2974b thru 0x3e32261 (0x8b17 bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2147 punch from 0x3aae285 to 0x3abad7b, (0xcaf6 bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2148 read 0x54a5cb2 thru 0x54b14de (0xb82d bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2149 trunc from 0xaa43c59 to 0xc07e1bb 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2150 punch from 0x577df37 to 0x578ca94, (0xeb5d bytes) 2024-09-21T23:10:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:2151 trunc from 0xc07e1bb to 0x7795593 2024-09-21T23:10:35.528 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:35.524+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:35.541 INFO:teuthology.orchestra.run.smithi150.stdout:2153 punch from 0x354baa3 to 0x354d5fb, (0x1b58 bytes) 2024-09-21T23:10:35.541 INFO:teuthology.orchestra.run.smithi150.stdout:2154 read 0x198a449 thru 0x199443a (0x9ff2 bytes) 2024-09-21T23:10:35.541 INFO:teuthology.orchestra.run.smithi150.stdout:2156 read 0x1146396 thru 0x1153cea (0xd955 bytes) 2024-09-21T23:10:35.567 INFO:teuthology.orchestra.run.smithi150.stdout:2158 writesame 0xe6b6973 thru 0xe6b91d2 data_size 0x13(0x2860 bytes) 2024-09-21T23:10:35.568 INFO:teuthology.orchestra.run.smithi150.stdout:2159 punch from 0x8400ad to 0x843233, (0x3186 bytes) 2024-09-21T23:10:35.568 INFO:teuthology.orchestra.run.smithi150.stdout:2160 read 0x7b93a88 thru 0x7b9570c (0x1c85 bytes) 2024-09-21T23:10:35.568 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.568 INFO:teuthology.orchestra.run.smithi150.stdout:2163 punch from 0xa56abf4 to 0xa56d7ef, (0x2bfb bytes) 2024-09-21T23:10:35.568 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.585 INFO:teuthology.orchestra.run.smithi150.stdout:2168 trunc from 0xe6b91d3 to 0x6a4d7a8 2024-09-21T23:10:35.599 INFO:teuthology.orchestra.run.smithi150.stdout:2169 writesame 0x3ba429 thru 0x3be264 data_size 0xe(0x3e3c bytes) 2024-09-21T23:10:35.602 INFO:teuthology.orchestra.run.smithi150.stdout:2170 punch from 0x1dae146 to 0x1db2c6a, (0x4b24 bytes) 2024-09-21T23:10:35.625 INFO:teuthology.orchestra.run.smithi150.stdout:2171 write 0xa1d1f00 thru 0xa1deabb (0xcbbc bytes) 2024-09-21T23:10:35.626 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:35.624+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:35.634 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:35.634 INFO:teuthology.orchestra.run.smithi150.stdout:2173 clone 21 order 24 su 131072 sc 6 2024-09-21T23:10:35.657 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:35.658 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:35.658 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:35.658 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:35.670 INFO:teuthology.orchestra.run.smithi150.stderr:marked out osd.7. 2024-09-21T23:10:35.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:35.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:10:35.758 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:35.760+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:35.827 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:35.828+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:35.835 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:35.835 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:35.836 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:35.836 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:35.836 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:35.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:35.938 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:35.940+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:36.023 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:36.023 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:36.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:36.023 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:36.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.023 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3817 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:36.023 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.1294963, 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020519229, 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3817 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:36.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892626+0000", 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892631+0000", 2024-09-21T23:10:36.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.57e-06 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892648+0000", 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7323e-05 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892663+0000", 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4610000000000001e-05 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908171+0000", 2024-09-21T23:10:36.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015508061 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908384+0000", 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000212945 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.913144+0000", 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047596000000000001 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3818 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 9895936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.12943946, 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016964216000000001, 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3818 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892682+0000", 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892688+0000", 2024-09-21T23:10:36.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7230000000000001e-06 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892707+0000", 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8423999999999998e-05 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892720+0000", 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3900000000000001e-05 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908948+0000", 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016227075000000001 2024-09-21T23:10:36.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909223+0000", 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027500799999999998 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909645+0000", 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000422963 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3821 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 15663104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.129383860000001, 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905422000000001, 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3821 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892738+0000", 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:36.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892744+0000", 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892763+0000", 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8916e-05 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892777+0000", 2024-09-21T23:10:36.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4216000000000001e-05 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909722+0000", 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016944846999999999 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909918+0000", 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019664199999999999 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.910642+0000", 2024-09-21T23:10:36.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072407799999999996 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6409 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253412164, 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016872036999999999, 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6409 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768710+0000", 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.877e-06 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768716+0000", 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768735+0000", 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8983000000000001e-05 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768750+0000", 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4246999999999999e-05 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785004+0000", 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016254497 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785251+0000", 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024662499999999999 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785581+0000", 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032997799999999998 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6411 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:36.040 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253298699999998, 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017242586000000001, 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6411 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:36.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768824+0000", 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6750000000000001e-06 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768847+0000", 2024-09-21T23:10:36.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3706999999999999e-05 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768878+0000", 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000003e-05 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768899+0000", 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0468000000000001e-05 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785675+0000", 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016776669000000001 2024-09-21T23:10:36.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785818+0000", 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014246199999999999 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786065+0000", 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024697500000000003 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6413 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.044 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253187883999999, 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017559430000000001, 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6413 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768934+0000", 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4929999999999999e-06 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768948+0000", 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:10:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768966+0000", 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8519999999999999e-05 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768980+0000", 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3725e-05 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786147+0000", 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017167094000000001 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786280+0000", 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013344499999999999 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786492+0000", 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021196600000000001 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6415 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:36.048 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253120009, 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905754999999999, 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6415 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769002+0000", 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769018+0000", 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6337999999999998e-05 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769036+0000", 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8042000000000001e-05 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769050+0000", 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786564+0000", 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017514538999999999 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786700+0000", 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786906+0000", 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000206689 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6418 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:36.052 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.253049502, 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018383022999999998, 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6418 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:36.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769072+0000", 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769078+0000", 2024-09-21T23:10:36.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4219999999999999e-06 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769094+0000", 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5815e-05 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769107+0000", 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3305000000000001e-05 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787013+0000", 2024-09-21T23:10:36.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017906090999999999 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787208+0000", 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019526600000000001 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787454+0000", 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024600099999999998 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.056 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6420 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252997972999999, 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018780271000000001, 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6420 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769124+0000", 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3799999999999996e-07 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769129+0000", 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769146+0000", 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198e-05 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769158+0000", 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1987999999999999e-05 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787568+0000", 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018410037000000001 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787697+0000", 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012926600000000001 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787903+0000", 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205692 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6421 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:36.060 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252947082999999, 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019118928, 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6421 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769179+0000", 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.78e-06 2024-09-21T23:10:36.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769194+0000", 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4576000000000001e-05 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769204+0000", 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0516999999999999e-05 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787991+0000", 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018786981000000001 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788130+0000", 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013833899999999999 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788293+0000", 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000162905 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6423 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252900405999998, 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019463588, 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6423 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769221+0000", 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999998e-07 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769226+0000", 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8799999999999999e-06 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769235+0000", 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8300000000000002e-06 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769241+0000", 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2580000000000001e-06 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788361+0000", 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019119873999999999 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788514+0000", 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152989 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788684+0000", 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016989899999999999 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6426 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252870978000001, 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019969425999999998, 2024-09-21T23:10:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6426 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769253+0000", 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5490000000000001e-06 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769260+0000", 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5650000000000004e-06 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769270+0000", 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9180000000000006e-06 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788787+0000", 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019516977000000001 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788981+0000", 2024-09-21T23:10:36.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019366300000000001 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789219+0000", 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023821399999999999 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6427 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252838609000001, 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020632774999999999, 2024-09-21T23:10:36.072 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6427 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:36.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769283+0000", 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1480000000000001e-06 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769288+0000", 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3819999999999997e-06 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769301+0000", 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2968e-05 2024-09-21T23:10:36.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769309+0000", 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1040000000000007e-06 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789376+0000", 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020067406999999999 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789621+0000", 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024461200000000001 2024-09-21T23:10:36.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789915+0000", 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029415399999999998 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6429 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252748935, 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021034428000000001, 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.076 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6429 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769374+0000", 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.906e-06 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769385+0000", 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1707e-05 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769410+0000", 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4102999999999999e-05 2024-09-21T23:10:36.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769429+0000", 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9110000000000002e-05 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790009+0000", 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020580751000000001 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790186+0000", 2024-09-21T23:10:36.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000176708 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790406+0000", 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022014300000000001 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6431 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252660793, 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021448724999999998, 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.080 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6431 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769461+0000", 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.403e-06 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769468+0000", 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4069999999999998e-06 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769487+0000", 2024-09-21T23:10:36.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769501+0000", 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4323e-05 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790548+0000", 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021047204 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790744+0000", 2024-09-21T23:10:36.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019580199999999999 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790909+0000", 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164709 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6433 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252600843, 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021886459, 2024-09-21T23:10:36.084 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6433 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:36.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769521+0000", 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769527+0000", 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7100000000000004e-06 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769544+0000", 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7229999999999999e-05 2024-09-21T23:10:36.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769558+0000", 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3567000000000001e-05 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790998+0000", 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021440516999999999 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791177+0000", 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017838899999999999 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791406+0000", 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022983300000000001 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6436 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252545949999998, 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022294958, 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6436 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769576+0000", 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769594+0000", 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8136999999999999e-05 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769612+0000", 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8063e-05 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769626+0000", 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3706999999999999e-05 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791524+0000", 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021898407000000002 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791653+0000", 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129116 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791870+0000", 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021640200000000001 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6437 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252451867000001, 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022700130999999998, 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6437 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769671+0000", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6530000000000001e-06 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769680+0000", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8830000000000001e-06 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769701+0000", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0883e-05 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769718+0000", 2024-09-21T23:10:36.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6385999999999999e-05 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791966+0000", 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022248622999999999 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792154+0000", 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000187834 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792369+0000", 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021486899999999999 2024-09-21T23:10:36.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6440 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.252382555000001, 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.023189891000000001, 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6440 2024-09-21T23:10:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769740+0000", 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.327e-06 2024-09-21T23:10:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769745+0000", 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6760000000000004e-06 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769762+0000", 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7187e-05 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769776+0000", 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3233e-05 2024-09-21T23:10:36.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792416+0000", 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022640389 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792608+0000", 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019183899999999999 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792928+0000", 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032024000000000001 2024-09-21T23:10:36.099 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:27644 3.10 3:0811e7db:::rbd_data.11499b8b109.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.668307214, 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041757458999999997, 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 27644 2024-09-21T23:10:36.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353815+0000", 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.426e-06 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359473+0000", 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056580709999999998 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359492+0000", 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8978999999999999e-05 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359500+0000", 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0909999999999993e-06 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359556+0000", 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6317999999999998e-05 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359660+0000", 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103915 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.360026+0000", 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036547400000000002 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.370944+0000", 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010918323000000001 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395472+0000", 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024528372999999999 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395527+0000", 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4873999999999997e-05 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:36.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395571+0000", 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3615000000000002e-05 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:36.105 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:36.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:36.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:36.107 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:36.104+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:36.207 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:36.204+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:36.257 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:36.257 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:36.274 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:36.274 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:36.276 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:36.276 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:36.290 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:36.290 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:36.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:36.309 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:36.309 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:36.309 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:36.329 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:36.329 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:36.334 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:36.336+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:36.347 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:36.347 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:36.367 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:36.367 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:36.370 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:36.371 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:36.408 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:36.408+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:36.410 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:36.410 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:36.424 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:36.515 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:36.516+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:36.588 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:36.589 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:36.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:36.646 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:36.648+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:36.679 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:10:36.711 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:36.708+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:36.744 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:36.744 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:36.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:36.844 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:36.844+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:36.913 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:36.912+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:36.920 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:36.920 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:36.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:37.020 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:37.020+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:37.100 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:37.100 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:37.114 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:37.114 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:37.112+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:37.215 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:37.212+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:37.281 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:37.281 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:37.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:37.336 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:37.336+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:37.416 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:37.416+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:37.456 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:37.456 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:37.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:37.518 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:37.516+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:37.618 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:37.616+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:37.643 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:37.643 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:37.657 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:37.719 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:37.720+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:37.816 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:37.816 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:37.819 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:37.820+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:37.835 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:37.920 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:37.920+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:37.959 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-21T23:10:37.966 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:37.966 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:37.985 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:38.003 INFO:teuthology.orchestra.run.smithi150.stdout:2174 trunc from 0xa1deabc to 0xa39139b 2024-09-21T23:10:38.012 INFO:teuthology.orchestra.run.smithi150.stdout:2175 write 0xa2e9804 thru 0xa2f01d9 (0x69d6 bytes) 2024-09-21T23:10:38.021 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:38.020+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:38.028 INFO:teuthology.orchestra.run.smithi150.stdout:2176 trunc from 0xa39139b to 0x68264bf 2024-09-21T23:10:38.049 INFO:teuthology.orchestra.run.smithi150.stdout:2178 read 0x424d7d3 thru 0x4256eaf (0x96dd bytes) 2024-09-21T23:10:38.049 INFO:teuthology.orchestra.run.smithi150.stdout:2179 punch from 0x1247bdc to 0x1249561, (0x1985 bytes) 2024-09-21T23:10:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:2180 write 0xedb9961 thru 0xedc7431 (0xdad1 bytes) 2024-09-21T23:10:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:2182 trunc from 0xedc7432 to 0xc13965c 2024-09-21T23:10:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:2183 punch from 0x89272ed to 0x892fe43, (0x8b56 bytes) 2024-09-21T23:10:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:2184 read 0x10718b9 thru 0x10753d9 (0x3b21 bytes) 2024-09-21T23:10:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:2185 read 0xf29b22 thru 0xf38a7c (0xef5b bytes) 2024-09-21T23:10:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:2186 punch from 0x89d63f5 to 0x89d8bc6, (0x27d1 bytes) 2024-09-21T23:10:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:2187 read 0x711ee7c thru 0x712ad4d (0xbed2 bytes) 2024-09-21T23:10:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:2188 write 0x7919249 thru 0x79262d1 (0xd089 bytes) 2024-09-21T23:10:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:2189 writesame 0x58c6283 thru 0x58ca48d data_size 0x1d(0x420b bytes) 2024-09-21T23:10:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:2190 read 0xa5a9310 thru 0xa5add4b (0x4a3c bytes) 2024-09-21T23:10:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:2191 writesame 0xc332eb9 thru 0xc3360f4 data_size 0x1(0x323c bytes) 2024-09-21T23:10:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:2193 punch from 0x644db5d to 0x645337e, (0x5821 bytes) 2024-09-21T23:10:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:2194 read 0x956b7de thru 0x957130b (0x5b2e bytes) 2024-09-21T23:10:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:2196 read 0x9dd145b thru 0x9ddc97d (0xb523 bytes) 2024-09-21T23:10:38.124 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:38.124+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:2197 writesame 0x6950ee3 thru 0x6956f8f data_size 0x1(0x60ad bytes) 2024-09-21T23:10:38.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:2198 write 0x4ed1f2b thru 0x4edc889 (0xa95f bytes) 2024-09-21T23:10:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:2199 read 0x62029f4 thru 0x6205ed8 (0x34e5 bytes) 2024-09-21T23:10:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:2202 read 0xc095bdc thru 0xc09eba2 (0x8fc7 bytes) 2024-09-21T23:10:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:2204 write 0xc0bd8b5 thru 0xc0c7c0d (0xa359 bytes) 2024-09-21T23:10:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:2206 punch from 0xc258890 to 0xc25c7a5, (0x3f15 bytes) 2024-09-21T23:10:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:2207 trunc from 0xc3360f5 to 0x193f455 2024-09-21T23:10:38.223 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:38.224+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:38.257 INFO:teuthology.orchestra.run.smithi150.stdout:2208 writesame 0x3ee47e4 thru 0x3ef3ec6 data_size 0x1(0xf6e3 bytes) 2024-09-21T23:10:38.277 INFO:teuthology.orchestra.run.smithi150.stdout:2209 writesame 0xca083ad thru 0xca0d46e data_size 0x1(0x50c2 bytes) 2024-09-21T23:10:38.277 INFO:teuthology.orchestra.run.smithi150.stdout:2210 write 0x84e2f01 thru 0x84e3f78 (0x1078 bytes) 2024-09-21T23:10:38.279 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:38.279 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:38.282 INFO:teuthology.orchestra.run.smithi150.stdout:2211 read 0x4032527 thru 0x403d2b7 (0xad91 bytes) 2024-09-21T23:10:38.283 INFO:teuthology.orchestra.run.smithi150.stdout:2212 writesame 0x6c30583 thru 0x6c327f4 data_size 0x1(0x2272 bytes) 2024-09-21T23:10:38.289 INFO:teuthology.orchestra.run.smithi150.stdout:2215 write 0x789ed03 thru 0x78aac8e (0xbf8c bytes) 2024-09-21T23:10:38.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:38.295 INFO:teuthology.orchestra.run.smithi150.stdout:2216 read 0x56b0de8 thru 0x56ba456 (0x966f bytes) 2024-09-21T23:10:38.306 INFO:teuthology.orchestra.run.smithi150.stdout:2217 trunc from 0xca0d46f to 0x2f60f93 2024-09-21T23:10:38.321 INFO:teuthology.orchestra.run.smithi150.stdout:2218 read 0x63bd04 thru 0x64a7e3 (0xeae0 bytes) 2024-09-21T23:10:38.322 INFO:teuthology.orchestra.run.smithi150.stdout:2219 read 0x29e7392 thru 0x29ea483 (0x30f2 bytes) 2024-09-21T23:10:38.322 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.324 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:38.324+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:38.339 INFO:teuthology.orchestra.run.smithi150.stdout:2221 writesame 0xce409c6 thru 0xce48bb0 data_size 0x1(0x81eb bytes) 2024-09-21T23:10:38.340 INFO:teuthology.orchestra.run.smithi150.stdout:2222 writesame 0xd56820 thru 0xd5fd4e data_size 0x1(0x952f bytes) 2024-09-21T23:10:38.361 INFO:teuthology.orchestra.run.smithi150.stdout:2224 punch from 0x4f0ee59 to 0x4f19f3e, (0xb0e5 bytes) 2024-09-21T23:10:38.361 INFO:teuthology.orchestra.run.smithi150.stdout:2225 read 0x905f62f thru 0x9065417 (0x5de9 bytes) 2024-09-21T23:10:38.362 INFO:teuthology.orchestra.run.smithi150.stdout:2226 read 0xb56cdd1 thru 0xb56ce4a (0x7a bytes) 2024-09-21T23:10:38.362 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.362 INFO:teuthology.orchestra.run.smithi150.stdout:2229 read 0x154ec4 thru 0x15f858 (0xa995 bytes) 2024-09-21T23:10:38.363 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.363 INFO:teuthology.orchestra.run.smithi150.stdout:2231 read 0x687ae34 thru 0x6881e6b (0x7038 bytes) 2024-09-21T23:10:38.363 INFO:teuthology.orchestra.run.smithi150.stdout:2232 punch from 0x66c943f to 0x66cdcb5, (0x4876 bytes) 2024-09-21T23:10:38.363 INFO:teuthology.orchestra.run.smithi150.stdout:2234 write 0x14be015 thru 0x14c9a50 (0xba3c bytes) 2024-09-21T23:10:38.370 INFO:teuthology.orchestra.run.smithi150.stdout:2235 trunc from 0xce48bb1 to 0x92ead42 2024-09-21T23:10:38.372 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:38.375 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.385 INFO:teuthology.orchestra.run.smithi150.stdout:2239 trunc from 0x92ead42 to 0xaa9625 2024-09-21T23:10:38.414 INFO:teuthology.orchestra.run.smithi150.stdout:2241 write 0xd5edcdf thru 0xd5f27e0 (0x4b02 bytes) 2024-09-21T23:10:38.419 INFO:teuthology.orchestra.run.smithi150.stdout:2243 trunc from 0xd5f27e1 to 0xe8f1417 2024-09-21T23:10:38.423 INFO:teuthology.orchestra.run.smithi150.stdout:2244 writesame 0xb8f3472 thru 0xb8f9a65 data_size 0x6(0x65f4 bytes) 2024-09-21T23:10:38.425 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:38.424+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:38.427 INFO:teuthology.orchestra.run.smithi150.stdout:2245 read 0x23764b1 thru 0x237c720 (0x6270 bytes) 2024-09-21T23:10:38.427 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.427 INFO:teuthology.orchestra.run.smithi150.stdout:2247 punch from 0x9ebccc7 to 0x9ec4a29, (0x7d62 bytes) 2024-09-21T23:10:38.428 INFO:teuthology.orchestra.run.smithi150.stdout:2248 writesame 0x307d423 thru 0x3088cc9 data_size 0x7(0xb8a7 bytes) 2024-09-21T23:10:38.432 INFO:teuthology.orchestra.run.smithi150.stdout:2249 read 0x4a6d1b1 thru 0x4a6e8c5 (0x1715 bytes) 2024-09-21T23:10:38.432 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.432 INFO:teuthology.orchestra.run.smithi150.stdout:2251 punch from 0x106dc3c to 0x107016b, (0x252f bytes) 2024-09-21T23:10:38.438 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:38.438 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:38.438 INFO:teuthology.orchestra.run.smithi150.stdout:2252 trunc from 0xe8f1417 to 0x9d9455d 2024-09-21T23:10:38.449 INFO:teuthology.orchestra.run.smithi150.stdout:2253 read 0x5dbb9cc thru 0x5dbe2a0 (0x28d5 bytes) 2024-09-21T23:10:38.449 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.450 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.450 INFO:teuthology.orchestra.run.smithi150.stdout:2256 read 0x58bb794 thru 0x58c252c (0x6d99 bytes) 2024-09-21T23:10:38.450 INFO:teuthology.orchestra.run.smithi150.stdout:2257 read 0x50b92dc thru 0x50bec0e (0x5933 bytes) 2024-09-21T23:10:38.451 INFO:teuthology.orchestra.run.smithi150.stdout:2258 punch from 0x2825936 to 0x282946e, (0x3b38 bytes) 2024-09-21T23:10:38.451 INFO:teuthology.orchestra.run.smithi150.stdout:2259 read 0x9c76253 thru 0x9c80536 (0xa2e4 bytes) 2024-09-21T23:10:38.451 INFO:teuthology.orchestra.run.smithi150.stdout:2260 punch from 0x89eb5ab to 0x89f3431, (0x7e86 bytes) 2024-09-21T23:10:38.451 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.451 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.451 INFO:teuthology.orchestra.run.smithi150.stdout:2264 writesame 0x34e6a1f thru 0x34f4e68 data_size 0x1(0xe44a bytes) 2024-09-21T23:10:38.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:38.471 INFO:teuthology.orchestra.run.smithi150.stdout:2265 punch from 0x7891d78 to 0x78a01d5, (0xe45d bytes) 2024-09-21T23:10:38.478 INFO:teuthology.orchestra.run.smithi150.stdout:2266 writesame 0xae5d3d9 thru 0xae5db38 data_size 0x10(0x760 bytes) 2024-09-21T23:10:38.489 INFO:teuthology.orchestra.run.smithi150.stdout:2267 trunc from 0xae5db39 to 0x254de8d 2024-09-21T23:10:38.506 INFO:teuthology.orchestra.run.smithi150.stdout:2268 writesame 0x2d4fa8b thru 0x2d52c0c data_size 0x1(0x3182 bytes) 2024-09-21T23:10:38.507 INFO:teuthology.orchestra.run.smithi150.stdout:2269 read 0xd9c409 thru 0xda55d8 (0x91d0 bytes) 2024-09-21T23:10:38.508 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.520 INFO:teuthology.orchestra.run.smithi150.stdout:2274 write 0x7a43356 thru 0x7a4a0f0 (0x6d9b bytes) 2024-09-21T23:10:38.524 INFO:teuthology.orchestra.run.smithi150.stdout:2276 write 0x3c4f1a4 thru 0x3c55b6c (0x69c9 bytes) 2024-09-21T23:10:38.526 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:38.528+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:38.532 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:38.533 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:38.535 INFO:teuthology.orchestra.run.smithi150.stdout:2277 write 0xb0ef51b thru 0xb0f0a96 (0x157c bytes) 2024-09-21T23:10:38.538 INFO:teuthology.orchestra.run.smithi150.stdout:2278 read 0x7b3bc04 thru 0x7b4b4ba (0xf8b7 bytes) 2024-09-21T23:10:38.538 INFO:teuthology.orchestra.run.smithi150.stdout:2279 punch from 0x19004fd to 0x190d4c9, (0xcfcc bytes) 2024-09-21T23:10:38.538 INFO:teuthology.orchestra.run.smithi150.stdout:2281 read 0x134dc9b thru 0x135ac37 (0xcf9d bytes) 2024-09-21T23:10:38.538 INFO:teuthology.orchestra.run.smithi150.stdout:2282 read 0x65d215d thru 0x65d4859 (0x26fd bytes) 2024-09-21T23:10:38.539 INFO:teuthology.orchestra.run.smithi150.stdout:2285 read 0x5cea1d5 thru 0x5cf6b9e (0xc9ca bytes) 2024-09-21T23:10:38.539 INFO:teuthology.orchestra.run.smithi150.stdout:2288 read 0x9181e7b thru 0x9187631 (0x57b7 bytes) 2024-09-21T23:10:38.549 INFO:teuthology.orchestra.run.smithi150.stdout:2289 trunc from 0xb0f0a97 to 0x4506725 2024-09-21T23:10:38.554 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:38.554 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:38.574 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:38.574 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:38.578 INFO:teuthology.orchestra.run.smithi150.stdout:2291 writesame 0xca0fd16 thru 0xca16ca1 data_size 0x76(0x6f8c bytes) 2024-09-21T23:10:38.583 INFO:teuthology.orchestra.run.smithi150.stdout:2292 trunc from 0xca16ca2 to 0x970ea96 2024-09-21T23:10:38.596 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:38.596 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:38.596 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:38.602 INFO:teuthology.orchestra.run.smithi150.stdout:2293 trunc from 0x970ea96 to 0x2563c6a 2024-09-21T23:10:38.607 INFO:teuthology.orchestra.run.smithi150.stdout:2294 punch from 0x22379a0 to 0x2242633, (0xac93 bytes) 2024-09-21T23:10:38.619 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:38.619 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:38.622 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4902 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.441969230000002, 2024-09-21T23:10:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030793945, 2024-09-21T23:10:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4902 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179180+0000", 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.37e-07 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179183+0000", 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9280000000000002e-06 2024-09-21T23:10:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179192+0000", 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7020000000000007e-06 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179198+0000", 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6780000000000003e-06 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201407+0000", 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022208301999999999 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.201500+0000", 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3231000000000002e-05 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.209974+0000", 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084735669999999996 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4903 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2359296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.441941255, 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031240032000000001, 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4903 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179208+0000", 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179211+0000", 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6630000000000002e-06 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179220+0000", 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8480000000000007e-06 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179226+0000", 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2650000000000002e-06 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210016+0000", 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030790128 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210066+0000", 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9713000000000003e-05 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210448+0000", 2024-09-21T23:10:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038190499999999998 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4904 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.441914027999999, 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031636297000000001, 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4904 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179235+0000", 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179238+0000", 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5349999999999999e-06 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179247+0000", 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6449999999999994e-06 2024-09-21T23:10:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179260+0000", 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3431999999999999e-05 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210480+0000", 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031220121 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210550+0000", 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0352000000000007e-05 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210871+0000", 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000320701 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4905 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.441878862999999, 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036964023999999998, 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4905 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179270+0000", 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179271+0000", 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179274+0000", 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1350000000000001e-06 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179283+0000", 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1779999999999992e-06 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179290+0000", 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210903+0000", 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.031613162 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.210976+0000", 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2591999999999999e-05 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216234+0000", 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052583309999999998 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:4906 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3735552~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.441849120000001, 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037394366999999998, 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 4906 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179300+0000", 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179303+0000", 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8219999999999998e-06 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179312+0000", 2024-09-21T23:10:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6419999999999996e-06 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.179319+0000", 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6449999999999999e-06 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216272+0000", 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036953142000000001 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216352+0000", 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0044999999999997e-05 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.216694+0000", 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034238399999999999 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5226 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.428452721999999, 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029942795000000001, 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5226 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192696+0000", 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192697+0000", 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7199999999999998e-07 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192701+0000", 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8e-06 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192712+0000", 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1156e-05 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192724+0000", 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2551999999999999e-05 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222110+0000", 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029385620000000001 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222191+0000", 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1520000000000006e-05 2024-09-21T23:10:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222639+0000", 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044747500000000002 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5227 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.428413381999999, 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030372671, 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5227 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192735+0000", 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192736+0000", 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192742+0000", 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2260000000000002e-06 2024-09-21T23:10:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192753+0000", 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0762e-05 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192761+0000", 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.14e-06 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222674+0000", 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029912979999999999 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.222754+0000", 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9816999999999998e-05 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223108+0000", 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035405799999999998 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5228 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1966080~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.428376468, 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0320895, 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5228 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192772+0000", 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192773+0000", 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192776+0000", 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2380000000000002e-06 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192787+0000", 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0764999999999999e-05 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192795+0000", 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3000000000000002e-06 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223154+0000", 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030358672999999999 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.223230+0000", 2024-09-21T23:10:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6268999999999998e-05 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224862+0000", 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001631635 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5229 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2424832~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:38.660 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.428342284999999, 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032496191000000001, 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5229 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192807+0000", 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5199999999999996e-07 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192810+0000", 2024-09-21T23:10:38.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192820+0000", 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0132000000000001e-05 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192829+0000", 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5900000000000008e-06 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224912+0000", 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032083410999999999 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.224985+0000", 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2455999999999999e-05 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225303+0000", 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031788999999999998 2024-09-21T23:10:38.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5230 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.428307611000001, 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033015543000000001, 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5230 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192841+0000", 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192842+0000", 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192845+0000", 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2040000000000002e-06 2024-09-21T23:10:38.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192856+0000", 2024-09-21T23:10:38.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1008e-05 2024-09-21T23:10:38.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192865+0000", 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225398+0000", 2024-09-21T23:10:38.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032533617000000001 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225489+0000", 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1038999999999994e-05 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225857+0000", 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000367597 2024-09-21T23:10:38.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5231 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.428273689000001, 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033349594000000003, 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5231 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192875+0000", 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192876+0000", 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192879+0000", 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2059999999999999e-06 2024-09-21T23:10:38.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192890+0000", 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0702e-05 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192898+0000", 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3299999999999999e-06 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225903+0000", 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033004622999999997 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.225981+0000", 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7990000000000004e-05 2024-09-21T23:10:38.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226225+0000", 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024411499999999999 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.677 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5232 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3801088~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.677 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:38.677 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.428239719, 2024-09-21T23:10:38.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033795809000000003, 2024-09-21T23:10:38.677 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.677 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.678 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.678 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5232 2024-09-21T23:10:38.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.678 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192909+0000", 2024-09-21T23:10:38.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192910+0000", 2024-09-21T23:10:38.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1800000000000005e-07 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192913+0000", 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1669999999999999e-06 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192924+0000", 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0692e-05 2024-09-21T23:10:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.192936+0000", 2024-09-21T23:10:38.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2414e-05 2024-09-21T23:10:38.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226252+0000", 2024-09-21T23:10:38.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03331626 2024-09-21T23:10:38.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226309+0000", 2024-09-21T23:10:38.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6733e-05 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.226705+0000", 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00039582499999999999 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5406 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.422726471000001, 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029727338999999998, 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.685 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5406 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198422+0000", 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198423+0000", 2024-09-21T23:10:38.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7100000000000001e-07 2024-09-21T23:10:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198427+0000", 2024-09-21T23:10:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:10:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198436+0000", 2024-09-21T23:10:38.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:38.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198446+0000", 2024-09-21T23:10:38.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6360000000000006e-06 2024-09-21T23:10:38.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227686+0000", 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029240289999999999 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.227764+0000", 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7676000000000006e-05 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228150+0000", 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038574300000000002 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5407 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:38.692 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.422692593000001, 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030154378999999999, 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5407 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198456+0000", 2024-09-21T23:10:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198457+0000", 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198460+0000", 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3409999999999999e-06 2024-09-21T23:10:38.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198470+0000", 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8460000000000003e-06 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198478+0000", 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7130000000000005e-06 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228216+0000", 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029738243000000001 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228290+0000", 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3461999999999996e-05 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228611+0000", 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032108100000000002 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5408 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:38.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.422661127000001, 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030920828000000001, 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5408 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198488+0000", 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2099999999999996e-07 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198491+0000", 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.988e-06 2024-09-21T23:10:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198501+0000", 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5719999999999992e-06 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.198508+0000", 2024-09-21T23:10:38.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2470000000000002e-06 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228720+0000", 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030211598999999999 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.228851+0000", 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013098299999999999 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.229409+0000", 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055781799999999996 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:5412 3.1e 3:79090407:::rbd_data.114912f5d0f.0000000000000002:7 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.420556118, 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.030113958, 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 5412 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200593+0000", 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4099999999999998e-07 2024-09-21T23:10:38.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200626+0000", 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2672999999999997e-05 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200637+0000", 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1248e-05 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.200646+0000", 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6559999999999998e-06 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230319+0000", 2024-09-21T23:10:38.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029672765 2024-09-21T23:10:38.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230378+0000", 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8696999999999997e-05 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:07.230707+0000", 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032917800000000001 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.711 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.712 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:38.712 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:38.712 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.546462873999999, 2024-09-21T23:10:38.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:38.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:38.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:38.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:38.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.720 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.722 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:38.722 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:38.722 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.380696022999999, 2024-09-21T23:10:38.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:38.722 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.722 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:38.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:38.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.723 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:38.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:38.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:38.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:38.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:38.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:38.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:38.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:38.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:38.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:38.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:38.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.732 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:21897 3.5 e37/21 3:a0a2d2ee:::rbd_data.1149c115bf92.0000000000000000:head v 37'78, mlcod=37'76)", 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.12897285, 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037373174000000002, 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492176+0000", 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.492178+0000", 2024-09-21T23:10:38.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9400000000000001e-06 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499469+0000", 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072913370000000002 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499506+0000", 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6412999999999998e-05 2024-09-21T23:10:38.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499524+0000", 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8097e-05 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499683+0000", 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000159631 2024-09-21T23:10:38.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.499726+0000", 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2716999999999998e-05 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.529473+0000", 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.029746356000000002 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:16.529549+0000", 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6682999999999997e-05 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:27646 3.b 3:d396a686:::rbd_data.11499b8b109.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:38.738 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.265429032, 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.032669003000000002, 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:38.739 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 27646 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:38.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355720+0000", 2024-09-21T23:10:38.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:38.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:38.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.355723+0000", 2024-09-21T23:10:38.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:10:38.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.360977+0000", 2024-09-21T23:10:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052540290000000003 2024-09-21T23:10:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361008+0000", 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0840000000000003e-05 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361028+0000", 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9771000000000001e-05 2024-09-21T23:10:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361182+0000", 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015453799999999999 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.361827+0000", 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064513600000000004 2024-09-21T23:10:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:38.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.363660+0000", 2024-09-21T23:10:38.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001833131 2024-09-21T23:10:38.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388259+0000", 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024598811000000002 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388305+0000", 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5479000000000003e-05 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:38.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388341+0000", 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6009e-05 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:21.388389+0000", 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8229000000000002e-05 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:38.747 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:38.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:10:38.749 INFO:teuthology.orchestra.run.smithi150.stdout:2296 write 0xa9dc88a thru 0xa9e5b66 (0x92dd bytes) 2024-09-21T23:10:38.749 INFO:teuthology.orchestra.run.smithi150.stdout:2299 write 0xb3a4066 thru 0xb3ad2eb (0x9286 bytes) 2024-09-21T23:10:38.750 INFO:teuthology.orchestra.run.smithi150.stdout:2301 write 0x32b1798 thru 0x32b4ee5 (0x374e bytes) 2024-09-21T23:10:38.750 INFO:teuthology.orchestra.run.smithi150.stdout:2303 read 0x5b9a860 thru 0x5ba17be (0x6f5f bytes) 2024-09-21T23:10:38.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:38.752 INFO:teuthology.orchestra.run.smithi150.stdout:2304 write 0xb4a2678 thru 0xb4ab171 (0x8afa bytes) 2024-09-21T23:10:38.752 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:38.752 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:2306 trunc from 0xb4ab172 to 0x9abc4bf 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:2307 punch from 0x3552afa to 0x35607d1, (0xdcd7 bytes) 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:2308 writesame 0x7897b60 thru 0x789ed13 data_size 0x26(0x71b4 bytes) 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:2310 read 0x3a3704d thru 0x3a3c51b (0x54cf bytes) 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:2311 punch from 0x3e48b92 to 0x3e49d20, (0x118e bytes) 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:2312 writesame 0x2ae6be0 thru 0x2af4dc0 data_size 0x4b(0xe1e1 bytes) 2024-09-21T23:10:38.754 INFO:teuthology.orchestra.run.smithi150.stdout:2313 read 0x1d38beb thru 0x1d41cc8 (0x90de bytes) 2024-09-21T23:10:38.755 INFO:teuthology.orchestra.run.smithi150.stdout:2314 punch from 0x1812552 to 0x181f609, (0xd0b7 bytes) 2024-09-21T23:10:38.755 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:38.756 INFO:teuthology.orchestra.run.smithi150.stdout:2316 read 0x3f3593b thru 0x3f3c16b (0x6831 bytes) 2024-09-21T23:10:38.756 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:38.752+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:38.756 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:38.757 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:38.757 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:38.758 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:38.759 INFO:teuthology.orchestra.run.smithi150.stdout:2322 write 0xa4fc4da thru 0xa4fe4b0 (0x1fd7 bytes) 2024-09-21T23:10:38.759 INFO:teuthology.orchestra.run.smithi150.stdout:2323 write 0x5cc4dd4 thru 0x5cc4f44 (0x171 bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2325 writesame 0x5b0490b thru 0x5b0c1e7 data_size 0x1(0x78dd bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2326 read 0x670d9df thru 0x671076d (0x2d8f bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2327 write 0x51c1d4c thru 0x51c6a7c (0x4d31 bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2329 write 0xe95f9be thru 0xe96554b (0x5b8e bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2330 punch from 0x42b8998 to 0x42b8bac, (0x214 bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2331 write 0x27bb419 thru 0x27be682 (0x326a bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2332 write 0xd444ad9 thru 0xd4522fa (0xd822 bytes) 2024-09-21T23:10:38.760 INFO:teuthology.orchestra.run.smithi150.stdout:2333 write 0x535c35e thru 0x5369bfa (0xd89d bytes) 2024-09-21T23:10:38.850 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:38.848+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:38.887 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:38.888 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:38.888 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:38.888 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:38.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:38.952 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:38.952+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:39.051 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:39.048+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:39.071 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:39.071 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:39.071 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:39.071 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:39.071 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:39.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:39.152 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:39.152+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:39.208 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:39.208 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:39.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:39.208 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5124 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.02027099, 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046086192999999998, 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5124 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186881+0000", 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.133e-06 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186887+0000", 2024-09-21T23:10:39.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7939999999999999e-06 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186904+0000", 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7193e-05 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186918+0000", 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223027+0000", 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036108460000000002 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223201+0000", 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017483900000000001 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.232966+0000", 2024-09-21T23:10:39.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097649039999999996 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5279 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018992349999998, 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045406334999999999, 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.214 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5279 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188160+0000", 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188167+0000", 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0700000000000001e-06 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188188+0000", 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-05 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188205+0000", 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7444000000000001e-05 2024-09-21T23:10:39.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233051+0000", 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044846496999999999 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233167+0000", 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115307 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233565+0000", 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039829399999999999 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5280 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018923153999999, 2024-09-21T23:10:39.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047857909999999997, 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5280 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188229+0000", 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.237e-06 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188235+0000", 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188254+0000", 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9015999999999999e-05 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188268+0000", 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.395e-05 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233608+0000", 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045339957 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233722+0000", 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011403700000000001 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236086+0000", 2024-09-21T23:10:39.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00236359 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5281 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018861653999998, 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048399091999999998, 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5281 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188291+0000", 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188298+0000", 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:10:39.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188315+0000", 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7819999999999999e-05 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188329+0000", 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3383e-05 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236130+0000", 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047800827999999997 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236240+0000", 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110745 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236689+0000", 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044823600000000002 2024-09-21T23:10:39.229 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5282 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018803775000002, 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048602487, 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.230 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5282 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188348+0000", 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188355+0000", 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3029999999999996e-06 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188373+0000", 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8056000000000001e-05 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188386+0000", 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3536999999999999e-05 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236748+0000", 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048361617000000003 2024-09-21T23:10:39.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236840+0000", 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2075000000000002e-05 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236950+0000", 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109755 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.234 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5283 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018660515999997, 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048844519000000003, 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.235 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5283 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:39.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188493+0000", 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3800000000000001e-06 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188503+0000", 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188526+0000", 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.304e-05 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188543+0000", 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7557000000000001e-05 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237020+0000", 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048476656 2024-09-21T23:10:39.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237187+0000", 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016670199999999999 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237335+0000", 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014857100000000001 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5284 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018578650000002, 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049117881000000002, 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.240 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5284 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188574+0000", 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188580+0000", 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188598+0000", 2024-09-21T23:10:39.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8304e-05 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188611+0000", 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3329000000000001e-05 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237402+0000", 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048790838000000003 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237549+0000", 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146897 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237690+0000", 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141383 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5285 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018521364000001, 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049392706000000001, 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5285 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.245 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188631+0000", 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0400000000000005e-07 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188636+0000", 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8329999999999999e-06 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188655+0000", 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8246000000000001e-05 2024-09-21T23:10:39.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188670+0000", 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5440000000000001e-05 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237748+0000", 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049077975000000003 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237890+0000", 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141953 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238022+0000", 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013235499999999999 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.249 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5286 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.018462261000003, 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049655168, 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5286 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.250 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188690+0000", 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1769999999999999e-06 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188696+0000", 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5269999999999998e-06 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188713+0000", 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7836e-05 2024-09-21T23:10:39.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188726+0000", 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238077+0000", 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049350964999999997 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238228+0000", 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015027000000000001 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238344+0000", 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116403 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.254 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5287 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.017692672999999, 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049431115999999997, 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5287 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189460+0000", 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189470+0000", 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189491+0000", 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0743000000000001e-05 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189503+0000", 2024-09-21T23:10:39.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2787e-05 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238395+0000", 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048891284 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238519+0000", 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124745 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238890+0000", 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037020400000000001 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5465 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.009896681000001, 2024-09-21T23:10:39.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045516793999999999, 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5465 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197256+0000", 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197262+0000", 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197280+0000", 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197294+0000", 2024-09-21T23:10:39.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3983e-05 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242417+0000", 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045122664999999999 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242550+0000", 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013329499999999999 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242771+0000", 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022120399999999999 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5466 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.009838623999997, 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045852049999999998, 2024-09-21T23:10:39.264 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5466 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197313+0000", 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197321+0000", 2024-09-21T23:10:39.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197338+0000", 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7824000000000001e-05 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197353+0000", 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.456e-05 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242853+0000", 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045500119999999998 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243010+0000", 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015682599999999999 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243165+0000", 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015463400000000001 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5467 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.009779381000001, 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046123232, 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5467 2024-09-21T23:10:39.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197373+0000", 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197378+0000", 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:10:39.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197397+0000", 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8793000000000001e-05 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197411+0000", 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3726999999999999e-05 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243249+0000", 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045838320000000002 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243376+0000", 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012760199999999999 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243495+0000", 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011848100000000001 2024-09-21T23:10:39.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5639 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.006967770000003, 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045343482999999997, 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5639 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200184+0000", 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200190+0000", 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:10:39.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200209+0000", 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8563000000000002e-05 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200223+0000", 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4173e-05 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.244967+0000", 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044743567999999997 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245103+0000", 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013634300000000001 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245527+0000", 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042370199999999998 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.279 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5640 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.006871912999998, 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045524901, 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5640 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200280+0000", 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200287+0000", 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200307+0000", 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0129999999999999e-05 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200321+0000", 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245576+0000", 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045255521 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245701+0000", 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012480100000000001 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245804+0000", 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102933 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5641 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.006809273999998, 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045750444000000001, 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5641 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.285 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200343+0000", 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200351+0000", 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9160000000000001e-06 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200369+0000", 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8490000000000001e-05 2024-09-21T23:10:39.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200383+0000", 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3234000000000001e-05 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245850+0000", 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045467883000000001 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245979+0000", 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128641 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246092+0000", 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113143 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.289 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5642 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.006749171000003, 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045975347999999999, 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5642 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200403+0000", 2024-09-21T23:10:39.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200409+0000", 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6570000000000002e-06 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200427+0000", 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8306000000000001e-05 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200441+0000", 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4042999999999999e-05 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246144+0000", 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045702495000000003 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246277+0000", 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133831 2024-09-21T23:10:39.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246377+0000", 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9835999999999998e-05 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5643 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:39.294 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.006691644999997, 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046409156, 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5643 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200460+0000", 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200466+0000", 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200483+0000", 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7776999999999999e-05 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200500+0000", 2024-09-21T23:10:39.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6852999999999999e-05 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246582+0000", 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046081471999999998 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246725+0000", 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143184 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246869+0000", 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014361699999999999 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5644 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.006631572000003, 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046601900000000002, 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.299 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5644 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200521+0000", 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200526+0000", 2024-09-21T23:10:39.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-06 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200544+0000", 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7410000000000001e-05 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200559+0000", 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5435999999999999e-05 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246920+0000", 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046360902000000002 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247055+0000", 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468200000000001 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247121+0000", 2024-09-21T23:10:39.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6712999999999996e-05 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5645 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.006572845999997, 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.04674565, 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.304 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5645 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200579+0000", 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2270000000000001e-06 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200585+0000", 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200605+0000", 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9675999999999999e-05 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200618+0000", 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509999999999999e-05 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247148+0000", 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046529451999999999 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247219+0000", 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0833000000000004e-05 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247324+0000", 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105122 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.308 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:39.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:39.310 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:39.272+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:39.374 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:39.372+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:39.436 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3.db408a71 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.71133122500000001, 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.71137955900000005, 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for peered", 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.437 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:10:39.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:10:39.441 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.441 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.441 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.441 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.441 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:10:39.441 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:10:39.441 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:39.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:39.494 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:39.496+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:39.573 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:39.574 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:39.574 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:39.574 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:39.574 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:39.575 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:39.572+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:39.587 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:39.676 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:39.676+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:39.705 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:10:39.724 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:39.724 INFO:teuthology.orchestra.run.smithi150.stdout:2335 write 0x483abfa thru 0x483e0a2 (0x34a9 bytes) 2024-09-21T23:10:39.727 INFO:teuthology.orchestra.run.smithi150.stdout:2336 write 0x728d50b thru 0x72978e0 (0xa3d6 bytes) 2024-09-21T23:10:39.737 INFO:teuthology.orchestra.run.smithi150.stdout:2337 trunc from 0xe96554c to 0xb627e8b 2024-09-21T23:10:39.752 INFO:teuthology.orchestra.run.smithi150.stdout:2338 read 0x4434f3f thru 0x4443427 (0xe4e9 bytes) 2024-09-21T23:10:39.753 INFO:teuthology.orchestra.run.smithi150.stdout:2341 read 0x26f349b thru 0x2702a47 (0xf5ad bytes) 2024-09-21T23:10:39.754 INFO:teuthology.orchestra.run.smithi150.stdout:2343 punch from 0x40fea74 to 0x410b8a6, (0xce32 bytes) 2024-09-21T23:10:39.754 INFO:teuthology.orchestra.run.smithi150.stdout:2344 punch from 0x810c66 to 0x81aedb, (0xa275 bytes) 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3817 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:39.760 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.868470684000002, 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020519229, 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3817 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892626+0000", 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892631+0000", 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.57e-06 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892648+0000", 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7323e-05 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892663+0000", 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4610000000000001e-05 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908171+0000", 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015508061 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908384+0000", 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000212945 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.913144+0000", 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047596000000000001 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3818 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 9895936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:39.764 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.868413844000003, 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016964216000000001, 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3818 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892681+0000", 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892682+0000", 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892688+0000", 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7230000000000001e-06 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892707+0000", 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8423999999999998e-05 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892720+0000", 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3900000000000001e-05 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908948+0000", 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016227075000000001 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909223+0000", 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027500799999999998 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909645+0000", 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000422963 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3821 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 15663104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:39.768 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.868358244, 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905422000000001, 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3821 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892738+0000", 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892744+0000", 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892763+0000", 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8916e-05 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892777+0000", 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4216000000000001e-05 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909722+0000", 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016944846999999999 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909918+0000", 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019664199999999999 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.910642+0000", 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072407799999999996 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.772 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6409 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.992386547999999, 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016872036999999999, 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6409 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768709+0000", 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768710+0000", 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.877e-06 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768716+0000", 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768735+0000", 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8983000000000001e-05 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768750+0000", 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4246999999999999e-05 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785004+0000", 2024-09-21T23:10:39.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016254497 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785251+0000", 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024662499999999999 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785581+0000", 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032997799999999998 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.776 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6411 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.992273084000001, 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017242586000000001, 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6411 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768824+0000", 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6750000000000001e-06 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768847+0000", 2024-09-21T23:10:39.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3706999999999999e-05 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768878+0000", 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000003e-05 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768899+0000", 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0468000000000001e-05 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785675+0000", 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016776669000000001 2024-09-21T23:10:39.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785818+0000", 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014246199999999999 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786065+0000", 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024697500000000003 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6413 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.780 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.992162268000001, 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017559430000000001, 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6413 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768934+0000", 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4929999999999999e-06 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768948+0000", 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768966+0000", 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8519999999999999e-05 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768980+0000", 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3725e-05 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786147+0000", 2024-09-21T23:10:39.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017167094000000001 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786280+0000", 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013344499999999999 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786492+0000", 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021196600000000001 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6415 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.784 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.992094392999999, 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905754999999999, 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6415 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769002+0000", 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769018+0000", 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6337999999999998e-05 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769036+0000", 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8042000000000001e-05 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769050+0000", 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786564+0000", 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017514538999999999 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786700+0000", 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786906+0000", 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000206689 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6418 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.788 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.992023885999998, 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018383022999999998, 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6418 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:39.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769072+0000", 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769078+0000", 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4219999999999999e-06 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769094+0000", 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5815e-05 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769107+0000", 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3305000000000001e-05 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787013+0000", 2024-09-21T23:10:39.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017906090999999999 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787208+0000", 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019526600000000001 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787454+0000", 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024600099999999998 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.792 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6420 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991972357000002, 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018780271000000001, 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6420 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769124+0000", 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3799999999999996e-07 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769129+0000", 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769146+0000", 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198e-05 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769158+0000", 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1987999999999999e-05 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787568+0000", 2024-09-21T23:10:39.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018410037000000001 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787697+0000", 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012926600000000001 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787903+0000", 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205692 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6421 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991921467000001, 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019118928, 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6421 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769179+0000", 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.78e-06 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769194+0000", 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4576000000000001e-05 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769204+0000", 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0516999999999999e-05 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787991+0000", 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018786981000000001 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788130+0000", 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013833899999999999 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788293+0000", 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000162905 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6423 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991874790000001, 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019463588, 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6423 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769221+0000", 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999998e-07 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769226+0000", 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8799999999999999e-06 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769235+0000", 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8300000000000002e-06 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769241+0000", 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2580000000000001e-06 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788361+0000", 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019119873999999999 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788514+0000", 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152989 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788684+0000", 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016989899999999999 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6426 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991845361999999, 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019969425999999998, 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6426 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769253+0000", 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5490000000000001e-06 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769260+0000", 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5650000000000004e-06 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769270+0000", 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9180000000000006e-06 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788787+0000", 2024-09-21T23:10:39.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019516977000000001 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788981+0000", 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019366300000000001 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789219+0000", 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023821399999999999 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.808 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6427 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991812993, 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020632774999999999, 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6427 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:39.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769283+0000", 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1480000000000001e-06 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769288+0000", 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3819999999999997e-06 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769301+0000", 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2968e-05 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769309+0000", 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1040000000000007e-06 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789376+0000", 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020067406999999999 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789621+0000", 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024461200000000001 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789915+0000", 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029415399999999998 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6429 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991723318999998, 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021034428000000001, 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6429 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769374+0000", 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.906e-06 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769385+0000", 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1707e-05 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769410+0000", 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4102999999999999e-05 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769429+0000", 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9110000000000002e-05 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790009+0000", 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020580751000000001 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790186+0000", 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000176708 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790406+0000", 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022014300000000001 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6431 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991635176999999, 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021448724999999998, 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6431 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769461+0000", 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.403e-06 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769468+0000", 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4069999999999998e-06 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769487+0000", 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769501+0000", 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4323e-05 2024-09-21T23:10:39.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790548+0000", 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021047204 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790744+0000", 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019580199999999999 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790909+0000", 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164709 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.820 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6433 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991575226999998, 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021886459, 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6433 2024-09-21T23:10:39.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769521+0000", 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:10:39.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769527+0000", 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7100000000000004e-06 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769544+0000", 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7229999999999999e-05 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769558+0000", 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3567000000000001e-05 2024-09-21T23:10:39.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790998+0000", 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021440516999999999 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791177+0000", 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017838899999999999 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791406+0000", 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022983300000000001 2024-09-21T23:10:39.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6436 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991520334000001, 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022294958, 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.825 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6436 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769576+0000", 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769594+0000", 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8136999999999999e-05 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769612+0000", 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8063e-05 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769626+0000", 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3706999999999999e-05 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791524+0000", 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021898407000000002 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791653+0000", 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129116 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791870+0000", 2024-09-21T23:10:39.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021640200000000001 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6437 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991426251, 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022700130999999998, 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6437 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769671+0000", 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6530000000000001e-06 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769680+0000", 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8830000000000001e-06 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769701+0000", 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0883e-05 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769718+0000", 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6385999999999999e-05 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791966+0000", 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022248622999999999 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792154+0000", 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000187834 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792369+0000", 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021486899999999999 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6440 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 31.991356938999999, 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.023189891000000001, 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:39.833 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6440 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769740+0000", 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.327e-06 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769745+0000", 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6760000000000004e-06 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769762+0000", 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7187e-05 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769776+0000", 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3233e-05 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792416+0000", 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022640389 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792608+0000", 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019183899999999999 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792928+0000", 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032024000000000001 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:27644 3.10 3:0811e7db:::rbd_data.11499b8b109.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.407281598000001, 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041757458999999997, 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:39.837 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 27644 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:39.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353815+0000", 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.426e-06 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359473+0000", 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056580709999999998 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359492+0000", 2024-09-21T23:10:39.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8978999999999999e-05 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359500+0000", 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0909999999999993e-06 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359556+0000", 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6317999999999998e-05 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:39.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359660+0000", 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103915 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.360026+0000", 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036547400000000002 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.370944+0000", 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010918323000000001 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395472+0000", 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024528372999999999 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395527+0000", 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4873999999999997e-05 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395571+0000", 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3615000000000002e-05 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.842 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.843 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.843 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:39.843 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:39.843 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:39.844 INFO:teuthology.orchestra.run.smithi150.stdout:2345 write 0xce0040c thru 0xce0530f (0x4f04 bytes) 2024-09-21T23:10:39.844 INFO:teuthology.orchestra.run.smithi150.stdout:2347 trunc from 0xce05310 to 0xb86add 2024-09-21T23:10:39.844 INFO:teuthology.orchestra.run.smithi150.stdout:2349 write 0x54aa2f7 thru 0x54b2912 (0x861c bytes) 2024-09-21T23:10:39.844 INFO:teuthology.orchestra.run.smithi150.stdout:2350 read 0x3fcba53 thru 0x3fcf17e (0x372c bytes) 2024-09-21T23:10:39.845 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:39.844+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:39.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:39.856 INFO:teuthology.orchestra.run.smithi150.stdout:2351 writesame 0xe5d188d thru 0xe5dc8da data_size 0x1(0xb04e bytes) 2024-09-21T23:10:39.864 INFO:teuthology.orchestra.run.smithi150.stdout:2352 trunc from 0xe5dc8db to 0xac29480 2024-09-21T23:10:39.883 INFO:teuthology.orchestra.run.smithi150.stdout:2354 writesame 0xe13547e thru 0xe13ebfd data_size 0x10(0x9780 bytes) 2024-09-21T23:10:39.883 INFO:teuthology.orchestra.run.smithi150.stdout:2355 write 0x45bf57f thru 0x45c192f (0x23b1 bytes) 2024-09-21T23:10:39.894 INFO:teuthology.orchestra.run.smithi150.stdout:2357 trunc from 0xe13ebfe to 0xaa25db3 2024-09-21T23:10:39.912 INFO:teuthology.orchestra.run.smithi150.stdout:2358 write 0xbee6c2e thru 0xbee6c5c (0x2f bytes) 2024-09-21T23:10:39.916 INFO:teuthology.orchestra.run.smithi150.stdout:2359 write 0x6f8468b thru 0x6f8f11d (0xaa93 bytes) 2024-09-21T23:10:39.921 INFO:teuthology.orchestra.run.smithi150.stdout:2360 read 0x748220b thru 0x748cff4 (0xadea bytes) 2024-09-21T23:10:39.921 INFO:teuthology.orchestra.run.smithi150.stdout:2361 punch from 0xb4f158a to 0xb4f5a0b, (0x4481 bytes) 2024-09-21T23:10:39.921 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:39.922 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:39.922 INFO:teuthology.orchestra.run.smithi150.stdout:2365 read 0x9df15a6 thru 0x9df755d (0x5fb8 bytes) 2024-09-21T23:10:39.924 INFO:teuthology.orchestra.run.smithi150.stdout:2366 trunc from 0xbee6c5d to 0xacbdd7e 2024-09-21T23:10:39.934 INFO:teuthology.orchestra.run.smithi150.stdout:2367 read 0xbbd184 thru 0xbbe9c0 (0x183d bytes) 2024-09-21T23:10:39.945 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:39.944+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:39.948 INFO:teuthology.orchestra.run.smithi150.stdout:2368 write 0xc79db6a thru 0xc7a9979 (0xbe10 bytes) 2024-09-21T23:10:39.955 INFO:teuthology.orchestra.run.smithi150.stdout:2370 punch from 0x376b470 to 0x376ba2f, (0x5bf bytes) 2024-09-21T23:10:39.955 INFO:teuthology.orchestra.run.smithi150.stdout:2371 punch from 0x70947c5 to 0x7099368, (0x4ba3 bytes) 2024-09-21T23:10:39.955 INFO:teuthology.orchestra.run.smithi150.stdout:2372 writesame 0x41f75c6 thru 0x41f83a9 data_size 0xe(0xde4 bytes) 2024-09-21T23:10:39.958 INFO:teuthology.orchestra.run.smithi150.stdout:2373 writesame 0x9764d8c thru 0x9767539 data_size 0x3(0x27ae bytes) 2024-09-21T23:10:39.967 INFO:teuthology.orchestra.run.smithi150.stdout:2375 punch from 0x51c7d6b to 0x51d281c, (0xaab1 bytes) 2024-09-21T23:10:39.967 INFO:teuthology.orchestra.run.smithi150.stdout:2378 writesame 0x21318b3 thru 0x213af7c data_size 0x1(0x96ca bytes) 2024-09-21T23:10:40.011 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.011 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.011 INFO:teuthology.orchestra.run.smithi150.stdout:2381 writesame 0x1613745 thru 0x161ccd8 data_size 0x6(0x9594 bytes) 2024-09-21T23:10:40.014 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:40.014 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:40.019 INFO:teuthology.orchestra.run.smithi150.stdout:2382 writesame 0xa8c28d7 thru 0xa8c9b1f data_size 0x1(0x7249 bytes) 2024-09-21T23:10:40.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:40.036 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.036 INFO:teuthology.orchestra.run.smithi150.stdout:2385 writesame 0x69ab684 thru 0x69b11b5 data_size 0x3(0x5b32 bytes) 2024-09-21T23:10:40.046 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:40.044+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:40.046 INFO:teuthology.orchestra.run.smithi150.stdout:2386 write 0xc583bad thru 0xc58cafb (0x8f4f bytes) 2024-09-21T23:10:40.052 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.052 INFO:teuthology.orchestra.run.smithi150.stdout:2388 writesame 0xac2abb4 thru 0xac2e249 data_size 0x33(0x3696 bytes) 2024-09-21T23:10:40.054 INFO:teuthology.orchestra.run.smithi150.stdout:2389 write 0x3bcb4b9 thru 0x3bcd678 (0x21c0 bytes) 2024-09-21T23:10:40.056 INFO:teuthology.orchestra.run.smithi150.stdout:2390 read 0xb2db65 thru 0xb34244 (0x66e0 bytes) 2024-09-21T23:10:40.057 INFO:teuthology.orchestra.run.smithi150.stdout:2391 read 0x52fac0b thru 0x52fced1 (0x22c7 bytes) 2024-09-21T23:10:40.058 INFO:teuthology.orchestra.run.smithi150.stdout:2393 punch from 0xafe515a to 0xaff10c9, (0xbf6f bytes) 2024-09-21T23:10:40.067 INFO:teuthology.orchestra.run.smithi150.stdout:2394 trunc from 0xc7a997a to 0x96b73e5 2024-09-21T23:10:40.091 INFO:teuthology.orchestra.run.smithi150.stdout:2398 read 0x409c308 thru 0x40a3b14 (0x780d bytes) 2024-09-21T23:10:40.092 INFO:teuthology.orchestra.run.smithi150.stdout:2399 write 0x94d9e2b thru 0x94e5076 (0xb24c bytes) 2024-09-21T23:10:40.098 INFO:teuthology.orchestra.run.smithi150.stdout:2400 writesame 0x3ccd01b thru 0x3ccd1e8 data_size 0xe(0x1ce bytes) 2024-09-21T23:10:40.123 INFO:teuthology.orchestra.run.smithi150.stdout:2402 trunc from 0x96b73e5 to 0x426255 2024-09-21T23:10:40.147 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:40.144+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:40.170 INFO:teuthology.orchestra.run.smithi150.stdout:2403 writesame 0x43da3ae thru 0x43e1103 data_size 0x2d(0x6d56 bytes) 2024-09-21T23:10:40.187 INFO:teuthology.orchestra.run.smithi150.stdout:2404 write 0x506c84d thru 0x5071577 (0x4d2b bytes) 2024-09-21T23:10:40.198 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:40.198 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:40.198 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:40.198 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:40.199 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:40.209 INFO:teuthology.orchestra.run.smithi150.stdout:2405 trunc from 0x5071578 to 0xc8f5900 2024-09-21T23:10:40.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:40.222 INFO:teuthology.orchestra.run.smithi150.stdout:2406 write 0xb749d70 thru 0xb74ad5b (0xfec bytes) 2024-09-21T23:10:40.227 INFO:teuthology.orchestra.run.smithi150.stdout:2407 writesame 0x2b94016 thru 0x2b9f98a data_size 0x1(0xb975 bytes) 2024-09-21T23:10:40.258 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:40.260+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:40.273 INFO:teuthology.orchestra.run.smithi150.stdout:2408 write 0xd590950 thru 0xd591ec8 (0x1579 bytes) 2024-09-21T23:10:40.279 INFO:teuthology.orchestra.run.smithi150.stdout:2409 read 0x22e11b thru 0x230fb2 (0x2e98 bytes) 2024-09-21T23:10:40.311 INFO:teuthology.orchestra.run.smithi150.stdout:2412 trunc from 0xd591ec9 to 0x12de90c 2024-09-21T23:10:40.344 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:40.344 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:40.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:40.344 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:40.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.159404887999997, 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:10:40.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:10:40.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:10:40.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.159301802000002, 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.349 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:10:40.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.159234453000003, 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.084047517000002, 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.083991748999999, 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.362 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:10:40.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:10:40.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.083960210000001, 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.370 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.083927461000002, 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:10:40.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.083900567999997, 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.083875294999999, 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.380 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:40.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:10:40.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:40.384 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.083844167999999, 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:40.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.083200812000001, 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.389 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.078014885000002, 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:10:40.393 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:10:40.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077993401999997, 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:10:40.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:10:40.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077969115999998, 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.402 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.406 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077951448999997, 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077927864000003, 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:10:40.411 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.415 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077911849000003, 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.419 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077898017999999, 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077866657000001, 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.424 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:10:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:10:40.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.077798235000003, 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:10:40.428 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:10:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:10:40.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.432 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:40.433 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:40.433 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:40.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:40.435 INFO:teuthology.orchestra.run.smithi150.stdout:2413 trunc from 0x12de90c to 0xc15088d 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:2415 writesame 0x3ceb0dc thru 0x3cec9c7 data_size 0xb(0x18ec bytes) 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:2416 write 0x1037aa4 thru 0x103c9c4 (0x4f21 bytes) 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:2419 trunc from 0xc15088d to 0xc6cd4d9 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:2420 writesame 0xa6649a4 thru 0xa66a60a data_size 0x53(0x5c67 bytes) 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:2424 write 0x263dcef thru 0x26442cf (0x65e1 bytes) 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:2425 punch from 0x5806732 to 0x5814510, (0xddde bytes) 2024-09-21T23:10:40.436 INFO:teuthology.orchestra.run.smithi150.stdout:2426 write 0xe2a4f63 thru 0xe2aa980 (0x5a1e bytes) 2024-09-21T23:10:40.437 INFO:teuthology.orchestra.run.smithi150.stdout:2427 read 0x5e1575 thru 0x5e8ad3 (0x755f bytes) 2024-09-21T23:10:40.438 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:40.432+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:40.453 INFO:teuthology.orchestra.run.smithi150.stdout:2428 trunc from 0xe2aa981 to 0x324264c 2024-09-21T23:10:40.473 INFO:teuthology.orchestra.run.smithi150.stdout:2429 read 0xc9f8b2 thru 0xcadf71 (0xe6c0 bytes) 2024-09-21T23:10:40.474 INFO:teuthology.orchestra.run.smithi150.stdout:2430 write 0x26d88b7 thru 0x26dae96 (0x25e0 bytes) 2024-09-21T23:10:40.476 INFO:teuthology.orchestra.run.smithi150.stdout:2431 read 0x1697cce thru 0x169a443 (0x2776 bytes) 2024-09-21T23:10:40.478 INFO:teuthology.orchestra.run.smithi150.stdout:2432 read 0xaa5f93 thru 0xab3b8a (0xdbf8 bytes) 2024-09-21T23:10:40.479 INFO:teuthology.orchestra.run.smithi150.stdout:2434 read 0x1082864 thru 0x108f4b3 (0xcc50 bytes) 2024-09-21T23:10:40.487 INFO:teuthology.orchestra.run.smithi150.stdout:2435 trunc from 0x324264c to 0x7d279d 2024-09-21T23:10:40.523 INFO:teuthology.orchestra.run.smithi150.stdout:2436 writesame 0x85f99b2 thru 0x85fdae6 data_size 0x1(0x4135 bytes) 2024-09-21T23:10:40.534 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:40.536+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:40.535 INFO:teuthology.orchestra.run.smithi150.stdout:2437 trunc from 0x85fdae7 to 0xe3db205 2024-09-21T23:10:40.543 INFO:teuthology.orchestra.run.smithi150.stdout:2439 writesame 0x2d46a13 thru 0x2d55e3a data_size 0x34(0xf428 bytes) 2024-09-21T23:10:40.571 INFO:teuthology.orchestra.run.smithi150.stdout:2440 trunc from 0xe3db205 to 0x1073e4f 2024-09-21T23:10:40.582 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:40.582 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:40.582 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:40.582 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:40.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:40.618 INFO:teuthology.orchestra.run.smithi150.stdout:2441 writesame 0xdd3feb1 thru 0xdd4f66a data_size 0x25(0xf7ba bytes) 2024-09-21T23:10:40.618 INFO:teuthology.orchestra.run.smithi150.stdout:2442 read 0x9a73a07 thru 0x9a8325b (0xf855 bytes) 2024-09-21T23:10:40.637 INFO:teuthology.orchestra.run.smithi150.stdout:2443 trunc from 0xdd4f66b to 0x2c5440a 2024-09-21T23:10:40.638 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:40.636+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:40.647 INFO:teuthology.orchestra.run.smithi150.stdout:2445 read 0x1e928a3 thru 0x1e995d6 (0x6d34 bytes) 2024-09-21T23:10:40.648 INFO:teuthology.orchestra.run.smithi150.stdout:2447 read 0x1aa8d22 thru 0x1ab3229 (0xa508 bytes) 2024-09-21T23:10:40.653 INFO:teuthology.orchestra.run.smithi150.stdout:2448 trunc from 0x2c5440a to 0xa4d02f 2024-09-21T23:10:40.683 INFO:teuthology.orchestra.run.smithi150.stdout:2449 writesame 0x683a4c5 thru 0x683fe35 data_size 0x1(0x5971 bytes) 2024-09-21T23:10:40.702 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T23:10:40.702 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:10:40.702 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:10:40.703 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-09-21T23:10:40.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T23:10:40.703 INFO:teuthology.orchestra.run.smithi150.stdout:2450 writesame 0xe1ed5cd thru 0xe1ed63d data_size 0x1(0x71 bytes) 2024-09-21T23:10:40.720 INFO:teuthology.orchestra.run.smithi150.stdout:2451 trunc from 0xe1ed63e to 0x4277bd6 2024-09-21T23:10:40.734 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.734 INFO:teuthology.orchestra.run.smithi150.stdout:2455 read 0x120a3e6 thru 0x121171c (0x7337 bytes) 2024-09-21T23:10:40.735 INFO:teuthology.orchestra.run.smithi150.stdout:2456 punch from 0x38c5a40 to 0x38c8675, (0x2c35 bytes) 2024-09-21T23:10:40.735 INFO:teuthology.orchestra.run.smithi150.stdout:2458 read 0x22c483 thru 0x2349d6 (0x8554 bytes) 2024-09-21T23:10:40.736 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:40.732+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:40.736 INFO:teuthology.orchestra.run.smithi150.stdout:2460 read 0x52238 thru 0x5e933 (0xc6fc bytes) 2024-09-21T23:10:40.737 INFO:teuthology.orchestra.run.smithi150.stdout:2462 write 0x1f022c8 thru 0x1f0a0e1 (0x7e1a bytes) 2024-09-21T23:10:40.739 INFO:teuthology.orchestra.run.smithi150.stdout:2463 punch from 0x402b6b to 0x40e07b, (0xb510 bytes) 2024-09-21T23:10:40.739 INFO:teuthology.orchestra.run.smithi150.stdout:2464 writesame 0x25c54 thru 0x266d0 data_size 0x5(0xa7d bytes) 2024-09-21T23:10:40.743 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.755 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:40.755 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:40.755 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:40.755 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:40.755 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:40.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:40.762 INFO:teuthology.orchestra.run.smithi150.stdout:2467 writesame 0xba7ebd2 thru 0xba8d7cb data_size 0x23(0xebfa bytes) 2024-09-21T23:10:40.768 INFO:teuthology.orchestra.run.smithi150.stdout:2468 trunc from 0xba8d7cc to 0x8d1fcb1 2024-09-21T23:10:40.776 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:40.776 INFO:teuthology.orchestra.run.smithi150.stdout:2470 punch from 0x8ca7e2d to 0x8cae55e, (0x6731 bytes) 2024-09-21T23:10:40.776 INFO:teuthology.orchestra.run.smithi150.stdout:2471 read 0x556be79 thru 0x556eace (0x2c56 bytes) 2024-09-21T23:10:40.788 INFO:teuthology.orchestra.run.smithi150.stdout:2472 trunc from 0x8d1fcb1 to 0x7316d5 2024-09-21T23:10:40.798 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.806 INFO:teuthology.orchestra.run.smithi150.stdout:2475 write 0x1a4aed6 thru 0x1a4ecda (0x3e05 bytes) 2024-09-21T23:10:40.810 INFO:teuthology.orchestra.run.smithi150.stdout:2476 read 0x10d3852 thru 0x10e2548 (0xecf7 bytes) 2024-09-21T23:10:40.811 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.811 INFO:teuthology.orchestra.run.smithi150.stdout:2479 punch from 0x189b8a6 to 0x189c143, (0x89d bytes) 2024-09-21T23:10:40.811 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.811 INFO:teuthology.orchestra.run.smithi150.stdout:2487 read 0x132e170 thru 0x13366fb (0x858c bytes) 2024-09-21T23:10:40.811 INFO:teuthology.orchestra.run.smithi150.stdout:2488 read 0x132bc8 thru 0x133506 (0x93f bytes) 2024-09-21T23:10:40.812 INFO:teuthology.orchestra.run.smithi150.stdout:2492 punch from 0x1115746 to 0x1117c17, (0x24d1 bytes) 2024-09-21T23:10:40.812 INFO:teuthology.orchestra.run.smithi150.stdout:2494 punch from 0x1997e6c to 0x19a2309, (0xa49d bytes) 2024-09-21T23:10:40.812 INFO:teuthology.orchestra.run.smithi150.stdout:2495 read 0xc58037 thru 0xc66b95 (0xeb5f bytes) 2024-09-21T23:10:40.812 INFO:teuthology.orchestra.run.smithi150.stdout:2496 read 0x143dafb thru 0x144acdc (0xd1e2 bytes) 2024-09-21T23:10:40.813 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.828 INFO:teuthology.orchestra.run.smithi150.stdout:2499 write 0x7a48323 thru 0x7a5782a (0xf508 bytes) 2024-09-21T23:10:40.833 INFO:teuthology.orchestra.run.smithi150.stdout:2503 writesame 0x56bd431 thru 0x56beddd data_size 0x7(0x19ad bytes) 2024-09-21T23:10:40.835 INFO:teuthology.orchestra.run.smithi150.stdout:2504 read 0x5b014c4 thru 0x5b0b28c (0x9dc9 bytes) 2024-09-21T23:10:40.836 INFO:teuthology.orchestra.run.smithi150.stdout:2506 read 0x732de3b thru 0x732f3ec (0x15b2 bytes) 2024-09-21T23:10:40.836 INFO:teuthology.orchestra.run.smithi150.stdout:2507 punch from 0x18163c8 to 0x1823779, (0xd3b1 bytes) 2024-09-21T23:10:40.836 INFO:teuthology.orchestra.run.smithi150.stdout:2508 punch from 0x4d8dc75 to 0x4d8df4b, (0x2d6 bytes) 2024-09-21T23:10:40.836 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:40.836+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:40.853 INFO:teuthology.orchestra.run.smithi150.stdout:2509 write 0xe857b51 thru 0xe864bc6 (0xd076 bytes) 2024-09-21T23:10:40.861 INFO:teuthology.orchestra.run.smithi150.stdout:2510 write 0xcfad6aa thru 0xcfb505a (0x79b1 bytes) 2024-09-21T23:10:40.875 INFO:teuthology.orchestra.run.smithi150.stdout:2511 trunc from 0xe864bc7 to 0x8458dfa 2024-09-21T23:10:40.889 INFO:teuthology.orchestra.run.smithi150.stdout:2512 read 0x74d237d thru 0x74db5c3 (0x9247 bytes) 2024-09-21T23:10:40.889 INFO:teuthology.orchestra.run.smithi150.stdout:2513 punch from 0x7406651 to 0x740e5b6, (0x7f65 bytes) 2024-09-21T23:10:40.889 INFO:teuthology.orchestra.run.smithi150.stdout:2514 write 0xee783 thru 0xf455d (0x5ddb bytes) 2024-09-21T23:10:40.892 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.893 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.893 INFO:teuthology.orchestra.run.smithi150.stdout:2518 write 0x825bce6 thru 0x82644f5 (0x8810 bytes) 2024-09-21T23:10:40.898 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.898 INFO:teuthology.orchestra.run.smithi150.stdout:2520 punch from 0x34d6047 to 0x34dd7fc, (0x77b5 bytes) 2024-09-21T23:10:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:2522 writesame 0x9760e82 thru 0x9767259 data_size 0xf(0x63d8 bytes) 2024-09-21T23:10:40.914 INFO:teuthology.orchestra.run.smithi150.stdout:2524 writesame 0x9cb0c91 thru 0x9cbc6e8 data_size 0x8(0xba58 bytes) 2024-09-21T23:10:40.916 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.916 INFO:teuthology.orchestra.run.smithi150.stdout:2527 write 0x114de4b thru 0x11531da (0x5390 bytes) 2024-09-21T23:10:40.921 INFO:teuthology.orchestra.run.smithi150.stdout:2528 trunc from 0x9cbc6e9 to 0xa26cc40 2024-09-21T23:10:40.926 INFO:teuthology.orchestra.run.smithi150.stdout:2529 write 0x745e8d2 thru 0x746b1c3 (0xc8f2 bytes) 2024-09-21T23:10:40.930 INFO:teuthology.orchestra.run.smithi150.stdout:2530 punch from 0x43be5af to 0x43cdae3, (0xf534 bytes) 2024-09-21T23:10:40.931 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.931 INFO:teuthology.orchestra.run.smithi150.stdout:2532 punch from 0x26c46e8 to 0x26c9ba0, (0x54b8 bytes) 2024-09-21T23:10:40.931 INFO:teuthology.orchestra.run.smithi150.stdout:2533 read 0xa11c354 thru 0xa128c46 (0xc8f3 bytes) 2024-09-21T23:10:40.931 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:40.931 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:40.932 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:40.932 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:40.932 INFO:teuthology.orchestra.run.smithi150.stdout:2534 write 0x77d0238 thru 0x77dbbf5 (0xb9be bytes) 2024-09-21T23:10:40.935 INFO:teuthology.orchestra.run.smithi150.stdout:2536 punch from 0xb3ea4a to 0xb4a649, (0xbbff bytes) 2024-09-21T23:10:40.935 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:40.935 INFO:teuthology.orchestra.run.smithi150.stdout:2539 read 0xfef190 thru 0xff55f1 (0x6462 bytes) 2024-09-21T23:10:40.935 INFO:teuthology.orchestra.run.smithi150.stdout:2542 write 0x8a3e02 thru 0x8ad565 (0x9764 bytes) 2024-09-21T23:10:40.936 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:40.936+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:40.937 INFO:teuthology.orchestra.run.smithi150.stdout:2543 writesame 0x9a43892 thru 0x9a4cf7e data_size 0x1b(0x96ed bytes) 2024-09-21T23:10:40.942 INFO:teuthology.orchestra.run.smithi150.stdout:2544 writesame 0x77f3d2f thru 0x77ff9d8 data_size 0x3e(0xbcaa bytes) 2024-09-21T23:10:40.944 INFO:teuthology.orchestra.run.smithi150.stdout:2545 read 0x79e264f thru 0x79eb3bc (0x8d6e bytes) 2024-09-21T23:10:40.944 INFO:teuthology.orchestra.run.smithi150.stdout:2547 read 0x1503b28 thru 0x150c6ef (0x8bc8 bytes) 2024-09-21T23:10:40.945 INFO:teuthology.orchestra.run.smithi150.stdout:2549 read 0x69c2381 thru 0x69cf4a6 (0xd126 bytes) 2024-09-21T23:10:40.946 INFO:teuthology.orchestra.run.smithi150.stdout:2551 punch from 0x2c307c3 to 0x2c33d07, (0x3544 bytes) 2024-09-21T23:10:40.946 INFO:teuthology.orchestra.run.smithi150.stdout:2552 punch from 0x19170db to 0x191b3e3, (0x4308 bytes) 2024-09-21T23:10:40.948 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:40.950 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:40.950 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:40.952 INFO:teuthology.orchestra.run.smithi150.stdout:2553 trunc from 0xa26cc40 to 0x638afe7 2024-09-21T23:10:40.962 INFO:teuthology.orchestra.run.smithi150.stdout:2555 punch from 0x420bdc to 0x4278ea, (0x6d0e bytes) 2024-09-21T23:10:40.971 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:40.971 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:40.973 INFO:teuthology.orchestra.run.smithi150.stdout:2556 writesame 0x76632dc thru 0x766fd87 data_size 0x22(0xcaac bytes) 2024-09-21T23:10:40.985 INFO:teuthology.orchestra.run.smithi150.stdout:2557 writesame 0x9fc1925 thru 0x9fc50c3 data_size 0x1(0x379f bytes) 2024-09-21T23:10:40.990 INFO:teuthology.orchestra.run.smithi150.stdout:2558 trunc from 0x9fc50c4 to 0x7e2ec46 2024-09-21T23:10:40.992 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:40.992 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:40.992 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:41.000 INFO:teuthology.orchestra.run.smithi150.stdout:2561 punch from 0x64f3113 to 0x6502493, (0xf380 bytes) 2024-09-21T23:10:41.000 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:41.005 INFO:teuthology.orchestra.run.smithi150.stdout:2563 trunc from 0x7e2ec46 to 0x4a42243 2024-09-21T23:10:41.018 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:41.018 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:41.035 INFO:teuthology.orchestra.run.smithi150.stdout:2564 trunc from 0x4a42243 to 0xcfd6c00 2024-09-21T23:10:41.036 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:41.036+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:41.041 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:41.041 INFO:teuthology.orchestra.run.smithi150.stdout:2566 punch from 0xb9380ff to 0xb946d3a, (0xec3b bytes) 2024-09-21T23:10:41.042 INFO:teuthology.orchestra.run.smithi150.stdout:2568 write 0x2759101 thru 0x2760c4d (0x7b4d bytes) 2024-09-21T23:10:41.043 INFO:teuthology.orchestra.run.smithi150.stdout:2569 write 0x133cd07 thru 0x134299a (0x5c94 bytes) 2024-09-21T23:10:41.045 INFO:teuthology.orchestra.run.smithi150.stdout:2570 punch from 0x55f148f to 0x55f7619, (0x618a bytes) 2024-09-21T23:10:41.045 INFO:teuthology.orchestra.run.smithi150.stdout:2571 writesame 0x8871d70 thru 0x888137d data_size 0x5(0xf60e bytes) 2024-09-21T23:10:41.049 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:41.049 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:41.054 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:41.054 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:41.054 INFO:teuthology.orchestra.run.smithi150.stdout:2576 write 0x4eb4fa9 thru 0x4ebbbac (0x6c04 bytes) 2024-09-21T23:10:41.057 INFO:teuthology.orchestra.run.smithi150.stdout:2577 write 0x83d3225 thru 0x83d70fb (0x3ed7 bytes) 2024-09-21T23:10:41.062 INFO:teuthology.orchestra.run.smithi150.stdout:2579 clone 22 order 23 su 4194304 sc 4 2024-09-21T23:10:41.071 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:41.071 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:41.074 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:41.074 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:41.089 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:41.089 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:41.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:41.144 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:41.144+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:41.239 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:41.240+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:41.263 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:41.263 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:41.277 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:41.339 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:41.336+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:41.401 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:41.401 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:41.415 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:41.440 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:41.440+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:41.541 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:41.540+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:41.579 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:41.579 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:41.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:41.642 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:41.640+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:41.717 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:41.717 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:41.731 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:41.732 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T23:10:41.737 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:10:41.755 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:41.752+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:41.813 INFO:teuthology.orchestra.run.smithi150.stdout:leaving image image_client.0-clone21 intact 2024-09-21T23:10:41.844 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:41.844+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:41.920 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:41.921 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:41.944 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:41.945 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:41.944+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:41.990 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-21T23:10:42.046 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:42.044+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:42.070 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:42.070 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:42.096 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:42.188 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:42.188+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:42.248 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:42.248+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:42.263 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:42.263 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:42.277 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:42.280 INFO:teuthology.orchestra.run.smithi150.stdout:2581 write 0xa80d9da thru 0xa813bc3 (0x61ea bytes) 2024-09-21T23:10:42.292 INFO:teuthology.orchestra.run.smithi150.stdout:2584 read 0xeb3125 thru 0xeb370e (0x5ea bytes) 2024-09-21T23:10:42.293 INFO:teuthology.orchestra.run.smithi150.stdout:2585 read 0xa34d8e7 thru 0xa357596 (0x9cb0 bytes) 2024-09-21T23:10:42.294 INFO:teuthology.orchestra.run.smithi150.stdout:2586 read 0xb16359e thru 0xb16b8f8 (0x835b bytes) 2024-09-21T23:10:42.294 INFO:teuthology.orchestra.run.smithi150.stdout:2587 write 0x3abad9b thru 0x3ac93a1 (0xe607 bytes) 2024-09-21T23:10:42.301 INFO:teuthology.orchestra.run.smithi150.stdout:2590 read 0xc219935 thru 0xc2205eb (0x6cb7 bytes) 2024-09-21T23:10:42.301 INFO:teuthology.orchestra.run.smithi150.stdout:2591 write 0x9b00fe7 thru 0x9b0ab4a (0x9b64 bytes) 2024-09-21T23:10:42.309 INFO:teuthology.orchestra.run.smithi150.stdout:2592 read 0x6e8fc89 thru 0x6e8fe10 (0x188 bytes) 2024-09-21T23:10:42.309 INFO:teuthology.orchestra.run.smithi150.stdout:2593 writesame 0x5956cbb thru 0x5961dc0 data_size 0x5b(0xb106 bytes) 2024-09-21T23:10:42.320 INFO:teuthology.orchestra.run.smithi150.stdout:2595 writesame 0xe6e4e5b thru 0xe6f17d5 data_size 0x21(0xc97b bytes) 2024-09-21T23:10:42.321 INFO:teuthology.orchestra.run.smithi150.stdout:2597 punch from 0x8657067 to 0x865cadc, (0x5a75 bytes) 2024-09-21T23:10:42.321 INFO:teuthology.orchestra.run.smithi150.stdout:2598 punch from 0xd804fc7 to 0xd813d5b, (0xed94 bytes) 2024-09-21T23:10:42.321 INFO:teuthology.orchestra.run.smithi150.stdout:2600 read 0xae954e5 thru 0xae9b72b (0x6247 bytes) 2024-09-21T23:10:42.321 INFO:teuthology.orchestra.run.smithi150.stdout:2601 write 0xa264786 thru 0xa273f1b (0xf796 bytes) 2024-09-21T23:10:42.326 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:42.326 INFO:teuthology.orchestra.run.smithi150.stdout:2603 writesame 0xc54fe56 thru 0xc55501c data_size 0x35(0x51c7 bytes) 2024-09-21T23:10:42.330 INFO:teuthology.orchestra.run.smithi150.stdout:2604 clone 23 order 19 su 8192 sc 7 2024-09-21T23:10:42.348 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:42.348+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:42.406 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:42.406 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:42.419 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:42.467 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:42.468+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:42.550 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:42.548+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:42.552 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:42.552 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:42.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:42.652 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:42.652+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:42.724 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:42.724 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:42.739 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:42.754 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:42.752+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:42.754 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:10:42.852 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:42.852+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:42.859 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone22 from 0xe6f17d6 (overlap 0xcfd6c00) to 0x67eb600 2024-09-21T23:10:42.913 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:42.914 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:42.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:42.975 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:42.976+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:43.054 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:43.052+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:43.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:43.075 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:43.109 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-21T23:10:43.154 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:43.152+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:43.227 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:43.227 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:43.237 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:43.237 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:43.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:43.254 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:43.254 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:43.271 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:43.271 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:43.287 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:43.288+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:43.288 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:43.288 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:43.289 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:43.307 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:43.307 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:43.326 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:43.326 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:43.347 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:43.347 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:43.350 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:43.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:43.355 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:43.356+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:43.369 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:43.369 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:43.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:43.369 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:43.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.369 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3817 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 8323072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.369 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.475370859000002, 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020519229, 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3817 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892624+0000", 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892626+0000", 2024-09-21T23:10:43.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892631+0000", 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.57e-06 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892648+0000", 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7323e-05 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892663+0000", 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4610000000000001e-05 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908171+0000", 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.015508061 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.908384+0000", 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000212945 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.913144+0000", 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047596000000000001 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:3821 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [read 15663104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.475258418999999, 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905422000000001, 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.374 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3821 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892737+0000", 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892738+0000", 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892744+0000", 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892763+0000", 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8916e-05 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.892777+0000", 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4216000000000001e-05 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909722+0000", 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016944846999999999 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.909918+0000", 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019664199999999999 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:06.910642+0000", 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072407799999999996 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.378 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6411 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.599173258999997, 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017242586000000001, 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6411 2024-09-21T23:10:43.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768822+0000", 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768824+0000", 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6750000000000001e-06 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768847+0000", 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3706999999999999e-05 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768878+0000", 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0630000000000003e-05 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768899+0000", 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0468000000000001e-05 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785675+0000", 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016776669000000001 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.785818+0000", 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014246199999999999 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786065+0000", 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024697500000000003 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6413 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.599062443000001, 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017559430000000001, 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.383 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6413 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768933+0000", 2024-09-21T23:10:43.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768934+0000", 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4929999999999999e-06 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768948+0000", 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768966+0000", 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8519999999999999e-05 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.768980+0000", 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3725e-05 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786147+0000", 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017167094000000001 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786280+0000", 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013344499999999999 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786492+0000", 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021196600000000001 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6415 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10420224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.387 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598994568000002, 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017905754999999999, 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6415 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769001+0000", 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769002+0000", 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769018+0000", 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6337999999999998e-05 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769036+0000", 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8042000000000001e-05 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769050+0000", 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786564+0000", 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017514538999999999 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786700+0000", 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.786906+0000", 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000206689 2024-09-21T23:10:43.391 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6418 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8847360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598924060999998, 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018383022999999998, 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.392 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6418 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769071+0000", 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769072+0000", 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769078+0000", 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4219999999999999e-06 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769094+0000", 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5815e-05 2024-09-21T23:10:43.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769107+0000", 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3305000000000001e-05 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787013+0000", 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.017906090999999999 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787208+0000", 2024-09-21T23:10:43.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019526600000000001 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787454+0000", 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024600099999999998 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6420 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598872532000001, 2024-09-21T23:10:43.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018780271000000001, 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6420 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:43.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769123+0000", 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769124+0000", 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3799999999999996e-07 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769129+0000", 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769146+0000", 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7198e-05 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769158+0000", 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1987999999999999e-05 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787568+0000", 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018410037000000001 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787697+0000", 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012926600000000001 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787903+0000", 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205692 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.400 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6421 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598821641999997, 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019118928, 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6421 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.401 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769174+0000", 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2999999999999999e-07 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769179+0000", 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.78e-06 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769194+0000", 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4576000000000001e-05 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769204+0000", 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0516999999999999e-05 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.787991+0000", 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.018786981000000001 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788130+0000", 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013833899999999999 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788293+0000", 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000162905 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6423 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598774964999997, 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019463588, 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6423 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769220+0000", 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769221+0000", 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999998e-07 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769226+0000", 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8799999999999999e-06 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769235+0000", 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8300000000000002e-06 2024-09-21T23:10:43.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769241+0000", 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2580000000000001e-06 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788361+0000", 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019119873999999999 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788514+0000", 2024-09-21T23:10:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000152989 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788684+0000", 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016989899999999999 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6426 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598745536999999, 2024-09-21T23:10:43.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019969425999999998, 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6426 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769250+0000", 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4000000000000002e-07 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769253+0000", 2024-09-21T23:10:43.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5490000000000001e-06 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769260+0000", 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5650000000000004e-06 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769270+0000", 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9180000000000006e-06 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788787+0000", 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019516977000000001 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.788981+0000", 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019366300000000001 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789219+0000", 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023821399999999999 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.413 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6427 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598713168000003, 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020632774999999999, 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.414 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6427 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769282+0000", 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769283+0000", 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1480000000000001e-06 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769288+0000", 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3819999999999997e-06 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769301+0000", 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2968e-05 2024-09-21T23:10:43.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769309+0000", 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1040000000000007e-06 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789376+0000", 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020067406999999999 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789621+0000", 2024-09-21T23:10:43.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024461200000000001 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.789915+0000", 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029415399999999998 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6429 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598623494000002, 2024-09-21T23:10:43.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021034428000000001, 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6429 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769372+0000", 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769374+0000", 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.906e-06 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769385+0000", 2024-09-21T23:10:43.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1707e-05 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769410+0000", 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4102999999999999e-05 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769429+0000", 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9110000000000002e-05 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790009+0000", 2024-09-21T23:10:43.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.020580751000000001 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790186+0000", 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000176708 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790406+0000", 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022014300000000001 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6431 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598535351999999, 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021448724999999998, 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6431 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.423 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769460+0000", 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769461+0000", 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.403e-06 2024-09-21T23:10:43.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769468+0000", 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4069999999999998e-06 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769487+0000", 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769501+0000", 2024-09-21T23:10:43.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4323e-05 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790548+0000", 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021047204 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790744+0000", 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019580199999999999 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790909+0000", 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164709 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6433 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598475401999998, 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021886459, 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.427 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6433 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769520+0000", 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769521+0000", 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769527+0000", 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7100000000000004e-06 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769544+0000", 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7229999999999999e-05 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769558+0000", 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3567000000000001e-05 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.790998+0000", 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021440516999999999 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791177+0000", 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017838899999999999 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791406+0000", 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022983300000000001 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.431 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6436 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598420509, 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022294958, 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6436 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769575+0000", 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769576+0000", 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769594+0000", 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8136999999999999e-05 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769612+0000", 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8063e-05 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769626+0000", 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3706999999999999e-05 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791524+0000", 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.021898407000000002 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791653+0000", 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129116 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791870+0000", 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021640200000000001 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.435 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6437 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598326426, 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022700130999999998, 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6437 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.436 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769669+0000", 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769671+0000", 2024-09-21T23:10:43.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6530000000000001e-06 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769680+0000", 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8830000000000001e-06 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769701+0000", 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0883e-05 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769718+0000", 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6385999999999999e-05 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.791966+0000", 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022248622999999999 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792154+0000", 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000187834 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792369+0000", 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021486899999999999 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:6440 3.10 3:0b4af118:::rbd_data.1149b00fdffd.0000000000000000:5 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.598257113999999, 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.023189891000000001, 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.440 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 6440 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769738+0000", 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769740+0000", 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.327e-06 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769745+0000", 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6760000000000004e-06 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769762+0000", 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7187e-05 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.769776+0000", 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3233e-05 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792416+0000", 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022640389 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792608+0000", 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019183899999999999 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.792928+0000", 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032024000000000001 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.444 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:27644 3.10 3:0811e7db:::rbd_data.11499b8b109.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.014181773000001, 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041757458999999997, 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 27644 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353815+0000", 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.426e-06 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359473+0000", 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056580709999999998 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359492+0000", 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8978999999999999e-05 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359500+0000", 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0909999999999993e-06 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359556+0000", 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6317999999999998e-05 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359660+0000", 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103915 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.360026+0000", 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036547400000000002 2024-09-21T23:10:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.370944+0000", 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010918323000000001 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395472+0000", 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024528372999999999 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395527+0000", 2024-09-21T23:10:43.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4873999999999997e-05 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395571+0000", 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3615000000000002e-05 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.640692595, 2024-09-21T23:10:43.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:10:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:10:43.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:10:43.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:43.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:10:43.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:10:43.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:43.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:10:43.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:10:43.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38323 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:40.230000+0000", 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.1379948099999999, 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.029941812000000002, 2024-09-21T23:10:43.458 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38323 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230000+0000", 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230000+0000", 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230004+0000", 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5360000000000001e-06 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230020+0000", 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5931999999999999e-05 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230056+0000", 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5961999999999999e-05 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230074+0000", 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8411e-05 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230182+0000", 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108124 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.230260+0000", 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7923000000000006e-05 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.235994+0000", 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005734014 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.259646+0000", 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.023651625999999999 2024-09-21T23:10:43.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.259676+0000", 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9835000000000001e-05 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.259778+0000", 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101746 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:40.259942+0000", 2024-09-21T23:10:43.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016470300000000001 2024-09-21T23:10:43.464 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.464 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.464 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.464 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.464 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:43.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:43.466 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:43.464+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:43.477 INFO:teuthology.orchestra.run.smithi150.stdout:2607 write 0x2dbe3bd thru 0x2dc8433 (0xa077 bytes) 2024-09-21T23:10:43.539 INFO:teuthology.orchestra.run.smithi150.stdout:2608 write 0xc165945 thru 0xc166f0f (0x15cb bytes) 2024-09-21T23:10:43.546 INFO:teuthology.orchestra.run.smithi150.stdout:2611 write 0xc404718 thru 0xc41006d (0xb956 bytes) 2024-09-21T23:10:43.566 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:43.564+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:43.637 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:43.637 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:43.637 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:43.637 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:43.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:43.667 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:43.664+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:43.760 INFO:teuthology.orchestra.run.smithi150.stdout:2612 trunc from 0xe6f17d6 to 0x664fe18 2024-09-21T23:10:43.767 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:43.764+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:43.782 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:43.782 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:43.782 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:43.783 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:43.783 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:43.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:43.808 INFO:teuthology.orchestra.run.smithi150.stdout:2613 write 0x4c6f43a thru 0x4c73e24 (0x49eb bytes) 2024-09-21T23:10:43.868 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:43.868+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:43.880 INFO:teuthology.orchestra.run.smithi150.stdout:2614 writesame 0xc64eaa3 thru 0xc650a9e data_size 0x17(0x1ffc bytes) 2024-09-21T23:10:43.892 INFO:teuthology.orchestra.run.smithi150.stdout:2615 trunc from 0xc650a9f to 0x6667e04 2024-09-21T23:10:43.912 INFO:teuthology.orchestra.run.smithi150.stdout:2618 write 0x6e3c641 thru 0x6e43ba6 (0x7566 bytes) 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:43.919 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.734757700999999, 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:10:43.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:10:43.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.734654614999997, 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.924 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:10:43.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:10:43.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.734587265999998, 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:43.931 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.659400329999997, 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:43.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:10:43.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:10:43.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.659344562000001, 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.659313023000003, 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.940 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:43.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:10:43.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:10:43.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.659280273999997, 2024-09-21T23:10:43.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:10:43.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.948 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.659253380999999, 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:10:43.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:10:43.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.659228108000001, 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.953 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.957 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.659196981000001, 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:43.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.658553625000003, 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:10:43.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653367697999997, 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.966 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653346214999999, 2024-09-21T23:10:43.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:10:43.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653321929000001, 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.978 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653304261999999, 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:10:43.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:43.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:10:43.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:10:43.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653280676999998, 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653264661999998, 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.987 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:10:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:10:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653250831000001, 2024-09-21T23:10:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653219470000003, 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:10:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:10:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.653151047999998, 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:10:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:10:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:10:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:10:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:44.004 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:44.006 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:44.004+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:44.006 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2620 read 0x890951 thru 0x89a84a (0x9efa bytes) 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2622 read 0x5cf5a53 thru 0x5d039df (0xdf8d bytes) 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2623 read 0x1bbfb20 thru 0x1bcca9e (0xcf7f bytes) 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2626 read 0x523b21a thru 0x523c386 (0x116d bytes) 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2627 trunc from 0x6e43ba7 to 0xddfefaf 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2628 read 0xd56a359 thru 0xd575d67 (0xba0f bytes) 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2629 punch from 0x41059ed to 0x410f738, (0x9d4b bytes) 2024-09-21T23:10:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:2630 punch from 0xc1ca8ae to 0xc1cde05, (0x3557 bytes) 2024-09-21T23:10:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:2632 write 0x2e57be2 thru 0x2e656c4 (0xdae3 bytes) 2024-09-21T23:10:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:2633 read 0x7bf29e7 thru 0x7bfacad (0x82c7 bytes) 2024-09-21T23:10:44.008 INFO:teuthology.orchestra.run.smithi150.stdout:2634 writesame 0x373f334 thru 0x374a946 data_size 0x3(0xb613 bytes) 2024-09-21T23:10:44.024 INFO:teuthology.orchestra.run.smithi150.stdout:2635 write 0xab70400 thru 0xab7b9c4 (0xb5c5 bytes) 2024-09-21T23:10:44.034 INFO:teuthology.orchestra.run.smithi150.stdout:2636 read 0x6cf9e53 thru 0x6cfbda3 (0x1f51 bytes) 2024-09-21T23:10:44.034 INFO:teuthology.orchestra.run.smithi150.stdout:2637 punch from 0x5e73d5c to 0x5e7be3e, (0x80e2 bytes) 2024-09-21T23:10:44.035 INFO:teuthology.orchestra.run.smithi150.stdout:2638 read 0x6dd6d30 thru 0x6de68db (0xfbac bytes) 2024-09-21T23:10:44.035 INFO:teuthology.orchestra.run.smithi150.stdout:2639 writesame 0xb0c68dd thru 0xb0d0861 data_size 0x1(0x9f85 bytes) 2024-09-21T23:10:44.051 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.051 INFO:teuthology.orchestra.run.smithi150.stdout:2642 read 0x991eb77 thru 0x991ec8a (0x114 bytes) 2024-09-21T23:10:44.051 INFO:teuthology.orchestra.run.smithi150.stdout:2643 write 0x1a98946 thru 0x1a9b820 (0x2edb bytes) 2024-09-21T23:10:44.067 INFO:teuthology.orchestra.run.smithi150.stdout:2644 trunc from 0xddfefaf to 0xd06f22d 2024-09-21T23:10:44.073 INFO:teuthology.orchestra.run.smithi150.stdout:2645 write 0x730162a thru 0x7309a4a (0x8421 bytes) 2024-09-21T23:10:44.082 INFO:teuthology.orchestra.run.smithi150.stdout:2649 write 0x176ac88 thru 0x1776801 (0xbb7a bytes) 2024-09-21T23:10:44.106 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:44.104+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:44.130 INFO:teuthology.orchestra.run.smithi150.stdout:2651 read 0x653c9f9 thru 0x654a8ee (0xdef6 bytes) 2024-09-21T23:10:44.133 INFO:teuthology.orchestra.run.smithi150.stdout:2652 read 0xaa40848 thru 0xaa4a12f (0x98e8 bytes) 2024-09-21T23:10:44.134 INFO:teuthology.orchestra.run.smithi150.stdout:2653 write 0xde807b thru 0xdf0cbf (0x8c45 bytes) 2024-09-21T23:10:44.164 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:44.165 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:44.170 INFO:teuthology.orchestra.run.smithi150.stdout:2654 write 0xebe7182 thru 0xebe847c (0x12fb bytes) 2024-09-21T23:10:44.182 INFO:teuthology.orchestra.run.smithi150.stdout:2656 trunc from 0xebe847d to 0xb30debe 2024-09-21T23:10:44.183 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:44.191 INFO:teuthology.orchestra.run.smithi150.stdout:2657 read 0xa008609 thru 0xa010d60 (0x8758 bytes) 2024-09-21T23:10:44.191 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.192 INFO:teuthology.orchestra.run.smithi150.stdout:2659 punch from 0x59fc0a9 to 0x5a068ee, (0xa845 bytes) 2024-09-21T23:10:44.196 INFO:teuthology.orchestra.run.smithi150.stdout:2662 trunc from 0xb30debe to 0x825db2c 2024-09-21T23:10:44.207 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:44.204+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:44.209 INFO:teuthology.orchestra.run.smithi150.stdout:2664 write 0x5aa2e5 thru 0x5b746e (0xd18a bytes) 2024-09-21T23:10:44.266 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.266 INFO:teuthology.orchestra.run.smithi150.stdout:2666 punch from 0x2f3695b to 0x2f43c3b, (0xd2e0 bytes) 2024-09-21T23:10:44.266 INFO:teuthology.orchestra.run.smithi150.stdout:2667 write 0x7dbf088 thru 0x7dc1ffc (0x2f75 bytes) 2024-09-21T23:10:44.270 INFO:teuthology.orchestra.run.smithi150.stdout:2668 punch from 0x195fac2 to 0x196b4fa, (0xba38 bytes) 2024-09-21T23:10:44.270 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.270 INFO:teuthology.orchestra.run.smithi150.stdout:2671 read 0x80af4e6 thru 0x80b32fc (0x3e17 bytes) 2024-09-21T23:10:44.270 INFO:teuthology.orchestra.run.smithi150.stdout:2673 read 0x1343fd5 thru 0x134dfa5 (0x9fd1 bytes) 2024-09-21T23:10:44.273 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.288 INFO:teuthology.orchestra.run.smithi150.stdout:2675 write 0xeb11ed4 thru 0xeb19504 (0x7631 bytes) 2024-09-21T23:10:44.303 INFO:teuthology.orchestra.run.smithi150.stdout:2676 trunc from 0xeb19505 to 0xa6efaec 2024-09-21T23:10:44.308 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:44.308+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:44.316 INFO:teuthology.orchestra.run.smithi150.stdout:2677 read 0x530480f thru 0x5312016 (0xd808 bytes) 2024-09-21T23:10:44.330 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:44.330 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:44.330 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:44.330 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:44.330 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:44.333 INFO:teuthology.orchestra.run.smithi150.stdout:2678 writesame 0xe3991ab thru 0xe39ad4d data_size 0x5(0x1ba3 bytes) 2024-09-21T23:10:44.344 INFO:teuthology.orchestra.run.smithi150.stdout:2679 trunc from 0xe39ad4e to 0x65eb014 2024-09-21T23:10:44.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:44.363 INFO:teuthology.orchestra.run.smithi150.stdout:2680 read 0x16b83d8 thru 0x16c411e (0xbd47 bytes) 2024-09-21T23:10:44.365 INFO:teuthology.orchestra.run.smithi150.stdout:2681 read 0x62a2bd8 thru 0x62a445d (0x1886 bytes) 2024-09-21T23:10:44.377 INFO:teuthology.orchestra.run.smithi150.stdout:2682 writesame 0xad20b6f thru 0xad267f7 data_size 0x1(0x5c89 bytes) 2024-09-21T23:10:44.377 INFO:teuthology.orchestra.run.smithi150.stdout:2683 write 0x1bc1637 thru 0x1bcaa64 (0x942e bytes) 2024-09-21T23:10:44.409 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:44.408+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:44.416 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.416 INFO:teuthology.orchestra.run.smithi150.stdout:2685 read 0x12af679 thru 0x12bc3cf (0xcd57 bytes) 2024-09-21T23:10:44.427 INFO:teuthology.orchestra.run.smithi150.stdout:2686 writesame 0xb40bd9d thru 0xb4144f9 data_size 0x1(0x875d bytes) 2024-09-21T23:10:44.427 INFO:teuthology.orchestra.run.smithi150.stdout:2687 read 0xa65562a thru 0xa65d72b (0x8102 bytes) 2024-09-21T23:10:44.428 INFO:teuthology.orchestra.run.smithi150.stdout:2689 punch from 0x999828d to 0x999a87d, (0x25f0 bytes) 2024-09-21T23:10:44.428 INFO:teuthology.orchestra.run.smithi150.stdout:2691 write 0x1fe0ed5 thru 0x1fe8491 (0x75bd bytes) 2024-09-21T23:10:44.469 INFO:teuthology.orchestra.run.smithi150.stdout:2692 read 0xa17c98 thru 0xa1a0bf (0x2428 bytes) 2024-09-21T23:10:44.470 INFO:teuthology.orchestra.run.smithi150.stdout:2693 punch from 0x229e096 to 0x22ab645, (0xd5af bytes) 2024-09-21T23:10:44.470 INFO:teuthology.orchestra.run.smithi150.stdout:2694 read 0x4c8032b thru 0x4c89bd8 (0x98ae bytes) 2024-09-21T23:10:44.471 INFO:teuthology.orchestra.run.smithi150.stdout:2697 read 0x3fec99 thru 0x408ec4 (0xa22c bytes) 2024-09-21T23:10:44.472 INFO:teuthology.orchestra.run.smithi150.stdout:2699 writesame 0x86d133d thru 0x86dd1f4 data_size 0x44(0xbeb8 bytes) 2024-09-21T23:10:44.491 INFO:teuthology.orchestra.run.smithi150.stdout:2700 trunc from 0xb4144fa to 0x7624736 2024-09-21T23:10:44.503 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.503 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89973008700000001, 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89968342000000001, 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89960935399999997, 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:10:44.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:10:44.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:10:44.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89954662799999996, 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:10:44.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89949467800000005, 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:10:44.520 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89928159200000002, 2024-09-21T23:10:44.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:10:44.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:10:44.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89739756699999995, 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89734951699999999, 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:10:44.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:10:44.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89730120999999996, 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.538 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:10:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:10:44.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89725399500000003, 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:10:44.546 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.897189766, 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:10:44.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89713785599999996, 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40186 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 442368~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89259025400000003, 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093234290999999997, 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40186 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610104+0000", 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610109+0000", 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2140000000000004e-06 2024-09-21T23:10:44.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610125+0000", 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5276000000000001e-05 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610139+0000", 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4477e-05 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703148+0000", 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093008570999999998 2024-09-21T23:10:44.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703236+0000", 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7868000000000006e-05 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703337+0000", 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101742 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89253556199999995, 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:10:44.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:10:44.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:10:44.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89248074700000002, 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:10:44.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:10:44.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89243302300000005, 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:10:44.568 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:10:44.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.89236613399999998, 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:10:44.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:10:44.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.88882085600000005, 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.88877411500000003, 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.581 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:10:44.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:10:44.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:10:44.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.88872759300000004, 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:10:44.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:10:44.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:44.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:44.590 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:44.590 INFO:teuthology.orchestra.run.smithi150.stdout:2701 write 0xe09cc26 thru 0xe0aa6ae (0xda89 bytes) 2024-09-21T23:10:44.591 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:44.592 INFO:teuthology.orchestra.run.smithi150.stdout:2702 write 0x8284baa thru 0x828ce3f (0x8296 bytes) 2024-09-21T23:10:44.592 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.592 INFO:teuthology.orchestra.run.smithi150.stdout:2704 write 0x86bb1cb thru 0x86c2ecc (0x7d02 bytes) 2024-09-21T23:10:44.592 INFO:teuthology.orchestra.run.smithi150.stdout:2706 trunc from 0xe0aa6af to 0x2ea01fb 2024-09-21T23:10:44.592 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:44.588+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:44.594 INFO:teuthology.orchestra.run.smithi150.stdout:2707 read 0x1e5447d thru 0x1e630ff (0xec83 bytes) 2024-09-21T23:10:44.596 INFO:teuthology.orchestra.run.smithi150.stdout:2708 writesame 0xa483fd thru 0xa517f1 data_size 0x7(0x93f5 bytes) 2024-09-21T23:10:44.639 INFO:teuthology.orchestra.run.smithi150.stdout:2710 read 0x814137 thru 0x81579d (0x1667 bytes) 2024-09-21T23:10:44.664 INFO:teuthology.orchestra.run.smithi150.stdout:2711 writesame 0xed7b48d thru 0xed7e881 data_size 0x1(0x33f5 bytes) 2024-09-21T23:10:44.664 INFO:teuthology.orchestra.run.smithi150.stdout:2713 write 0x4c2b8bf thru 0x4c2f667 (0x3da9 bytes) 2024-09-21T23:10:44.671 INFO:teuthology.orchestra.run.smithi150.stdout:2714 writesame 0xe1651ee thru 0xe16d4b9 data_size 0x16(0x82cc bytes) 2024-09-21T23:10:44.681 INFO:teuthology.orchestra.run.smithi150.stdout:2715 write 0xa1257d3 thru 0xa132d7f (0xd5ad bytes) 2024-09-21T23:10:44.692 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:44.692+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:44.713 INFO:teuthology.orchestra.run.smithi150.stdout:2716 trunc from 0xed7e882 to 0x18ce8c6 2024-09-21T23:10:44.735 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:44.735 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:44.735 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:44.735 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:44.741 INFO:teuthology.orchestra.run.smithi150.stdout:2717 trunc from 0x18ce8c6 to 0x52f8375 2024-09-21T23:10:44.746 INFO:teuthology.orchestra.run.smithi150.stdout:2718 read 0x14a9bf6 thru 0x14b2192 (0x859d bytes) 2024-09-21T23:10:44.748 INFO:teuthology.orchestra.run.smithi150.stdout:2719 read 0x10cb371 thru 0x10db312 (0xffa2 bytes) 2024-09-21T23:10:44.750 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.750 INFO:teuthology.orchestra.run.smithi150.stdout:2721 read 0x47bbd46 thru 0x47c9a86 (0xdd41 bytes) 2024-09-21T23:10:44.750 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.751 INFO:teuthology.orchestra.run.smithi150.stdout:2723 read 0x4d9eefd thru 0x4daa45b (0xb55f bytes) 2024-09-21T23:10:44.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:44.767 INFO:teuthology.orchestra.run.smithi150.stdout:2726 writesame 0xc88b059 thru 0xc891a83 data_size 0x1(0x6a2b bytes) 2024-09-21T23:10:44.767 INFO:teuthology.orchestra.run.smithi150.stdout:2727 punch from 0x50de350 to 0x50e2170, (0x3e20 bytes) 2024-09-21T23:10:44.767 INFO:teuthology.orchestra.run.smithi150.stdout:2728 write 0x157fd0e thru 0x1583486 (0x3779 bytes) 2024-09-21T23:10:44.770 INFO:teuthology.orchestra.run.smithi150.stdout:2730 punch from 0x7bc56d9 to 0x7bd5268, (0xfb8f bytes) 2024-09-21T23:10:44.770 INFO:teuthology.orchestra.run.smithi150.stdout:2731 punch from 0x2f72b68 to 0x2f76621, (0x3ab9 bytes) 2024-09-21T23:10:44.770 INFO:teuthology.orchestra.run.smithi150.stdout:2733 read 0xc50f1d6 thru 0xc51cdec (0xdc17 bytes) 2024-09-21T23:10:44.771 INFO:teuthology.orchestra.run.smithi150.stdout:2734 read 0x165c7c2 thru 0x1669fee (0xd82d bytes) 2024-09-21T23:10:44.771 INFO:teuthology.orchestra.run.smithi150.stdout:2736 write 0xa54326f thru 0xa549c37 (0x69c9 bytes) 2024-09-21T23:10:44.779 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:10:44.780 INFO:teuthology.orchestra.run.smithi150.stdout:2737 write 0xfa3e65 thru 0xfb383a (0xf9d6 bytes) 2024-09-21T23:10:44.793 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:44.792+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:44.841 INFO:teuthology.orchestra.run.smithi150.stdout:2740 write 0xa2d1a64 thru 0xa2d1baf (0x14c bytes) 2024-09-21T23:10:44.845 INFO:teuthology.orchestra.run.smithi150.stdout:2741 writesame 0xab2e3c4 thru 0xab2ea2e data_size 0x1(0x66b bytes) 2024-09-21T23:10:44.849 INFO:teuthology.orchestra.run.smithi150.stdout:2742 punch from 0xaf36fd6 to 0xaf39ed4, (0x2efe bytes) 2024-09-21T23:10:44.857 INFO:teuthology.orchestra.run.smithi150.stdout:2744 write 0xe6ed87c thru 0xe6efff6 (0x277b bytes) 2024-09-21T23:10:44.862 INFO:teuthology.orchestra.run.smithi150.stdout:2745 read 0xd676c05 thru 0xd68427a (0xd676 bytes) 2024-09-21T23:10:44.862 INFO:teuthology.orchestra.run.smithi150.stdout:2747 writesame 0x8090d65 thru 0x8095470 data_size 0x2(0x470c bytes) 2024-09-21T23:10:44.870 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.870 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.870 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.871 INFO:teuthology.orchestra.run.smithi150.stdout:2754 punch from 0x96a26bb to 0x96ab9cb, (0x9310 bytes) 2024-09-21T23:10:44.871 INFO:teuthology.orchestra.run.smithi150.stdout:2756 writesame 0xb3d9eed thru 0xb3e695d data_size 0x19(0xca71 bytes) 2024-09-21T23:10:44.883 INFO:teuthology.orchestra.run.smithi150.stdout:2759 write 0x1766998 thru 0x176bc58 (0x52c1 bytes) 2024-09-21T23:10:44.886 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.886 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:44.886 INFO:teuthology.orchestra.run.smithi150.stdout:2763 write 0x5dcceee thru 0x5ddb619 (0xe72c bytes) 2024-09-21T23:10:44.893 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:44.892+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:44.896 INFO:teuthology.orchestra.run.smithi150.stdout:2764 write 0xa0055e7 thru 0xa00efc2 (0x99dc bytes) 2024-09-21T23:10:44.906 INFO:teuthology.orchestra.run.smithi150.stdout:2766 writesame 0xcf1e118 thru 0xcf22bd8 data_size 0x1(0x4ac1 bytes) 2024-09-21T23:10:44.912 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:44.912 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:44.920 INFO:teuthology.orchestra.run.smithi150.stdout:2767 read 0xc1032e6 thru 0xc105fc9 (0x2ce4 bytes) 2024-09-21T23:10:44.920 INFO:teuthology.orchestra.run.smithi150.stdout:2768 write 0x42cdb01 thru 0x42d857e (0xaa7e bytes) 2024-09-21T23:10:44.933 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:44.934 INFO:teuthology.orchestra.run.smithi150.stdout:2769 read 0x93a112b thru 0x93ae49f (0xd375 bytes) 2024-09-21T23:10:44.934 INFO:teuthology.orchestra.run.smithi150.stdout:2770 clone 24 order 18 su 65536 sc 14 2024-09-21T23:10:44.995 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:44.996+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:45.051 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:45.051 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:45.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:45.051 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:45.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.051 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:34348 3.4 3:2118dc66:::rbd_data.114937d450a8.0000000000000012:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-21T23:10:45.051 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:31.241078+0000", 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.807986902, 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.033744523999999998, 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 34348 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241078+0000", 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241078+0000", 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241079+0000", 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5379999999999999e-06 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241086+0000", 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8079999999999999e-06 2024-09-21T23:10:45.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241100+0000", 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4265e-05 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241108+0000", 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8110000000000003e-06 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241182+0000", 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4031999999999999e-05 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.241278+0000", 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6206999999999996e-05 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 2", 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.246763+0000", 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054850089999999999 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.274689+0000", 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.027926116000000001 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.274725+0000", 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5231999999999997e-05 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.274766+0000", 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1872000000000002e-05 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:31.274822+0000", 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5634000000000003e-05 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35558 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 3604480~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992423+0000", 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.05664118, 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023680744, 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35558 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992423+0000", 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992423+0000", 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992423+0000", 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999999999999999e-07 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992425+0000", 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992428+0000", 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.163e-06 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992430+0000", 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4600000000000002e-06 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.015982+0000", 2024-09-21T23:10:45.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023552086999999999 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016029+0000", 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6650999999999998e-05 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016104+0000", 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5079000000000004e-05 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35559 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 3866624~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992433+0000", 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056631126999999, 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023939711999999998, 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35559 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992433+0000", 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992433+0000", 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992434+0000", 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992438+0000", 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-21T23:10:45.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992441+0000", 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992444+0000", 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5739999999999998e-06 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016139+0000", 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023695371999999999 2024-09-21T23:10:45.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016238+0000", 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8187e-05 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016373+0000", 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013548499999999999 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.064 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35560 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 4128768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992447+0000", 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056617025, 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024080621999999999, 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35560 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992447+0000", 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992447+0000", 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992448+0000", 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3099999999999999e-07 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992449+0000", 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.164e-06 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992452+0000", 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2679999999999999e-06 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992455+0000", 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4509999999999999e-06 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016404+0000", 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023949446999999999 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016453+0000", 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9177e-05 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016528+0000", 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4883999999999997e-05 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35577 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992458+0000", 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056606842000001, 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024344425999999999, 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.068 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35577 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992458+0000", 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992458+0000", 2024-09-21T23:10:45.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992458+0000", 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.23e-07 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992459+0000", 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0279999999999999e-06 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992462+0000", 2024-09-21T23:10:45.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2550000000000002e-06 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992465+0000", 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4159999999999998e-06 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016568+0000", 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024103493 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016661+0000", 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3357999999999996e-05 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016802+0000", 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140653 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35578 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992468+0000", 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056596824, 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024473577, 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35578 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992468+0000", 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992468+0000", 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992468+0000", 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992469+0000", 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.006e-06 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992472+0000", 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-21T23:10:45.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992474+0000", 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.429e-06 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016817+0000", 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024342321 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016862+0000", 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5188999999999997e-05 2024-09-21T23:10:45.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016941+0000", 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9328999999999999e-05 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35579 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992477+0000", 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056587123, 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024608748, 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.076 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35579 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992477+0000", 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992477+0000", 2024-09-21T23:10:45.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992478+0000", 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992479+0000", 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992482+0000", 2024-09-21T23:10:45.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.123e-06 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992484+0000", 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4700000000000001e-06 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.016958+0000", 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024473635000000001 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017004+0000", 2024-09-21T23:10:45.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5858e-05 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017086+0000", 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2427000000000002e-05 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35580 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992487+0000", 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056577194000001, 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024875099000000001, 2024-09-21T23:10:45.080 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35580 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992487+0000", 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992487+0000", 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992487+0000", 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-07 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992488+0000", 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992492+0000", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1319999999999998e-06 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992494+0000", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3810000000000002e-06 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017131+0000", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024636860999999999 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017221+0000", 2024-09-21T23:10:45.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0011000000000003e-05 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017362+0000", 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014153599999999999 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35581 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992497+0000", 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056567362999999, 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025024603999999999, 2024-09-21T23:10:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35581 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992497+0000", 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992497+0000", 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992497+0000", 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2100000000000001e-07 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992498+0000", 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992502+0000", 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2329999999999998e-06 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992511+0000", 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5920000000000007e-06 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017391+0000", 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024879645999999998 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017445+0000", 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4635000000000001e-05 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017522+0000", 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6291999999999994e-05 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35582 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992515+0000", 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056549564000001, 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025294882000000001, 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35582 2024-09-21T23:10:45.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992515+0000", 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992515+0000", 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992515+0000", 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:45.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992516+0000", 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1599999999999999e-06 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992520+0000", 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.534e-06 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992529+0000", 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2099999999999999e-06 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017572+0000", 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025043009000000001 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017667+0000", 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5155000000000006e-05 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017810+0000", 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014256499999999999 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35583 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992532+0000", 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056532017, 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025407814000000001, 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35583 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992532+0000", 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992532+0000", 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992533+0000", 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3900000000000001e-07 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992536+0000", 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7850000000000002e-06 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992540+0000", 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.822e-06 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992543+0000", 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5569999999999998e-06 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017823+0000", 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025280418999999998 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017870+0000", 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6347999999999997e-05 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017940+0000", 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0643999999999997e-05 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.097 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35584 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992546+0000", 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056518229, 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025681011, 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35584 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.098 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992546+0000", 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992546+0000", 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992546+0000", 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.34e-07 2024-09-21T23:10:45.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992548+0000", 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1039999999999999e-06 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992551+0000", 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.208e-06 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992554+0000", 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2440000000000002e-06 2024-09-21T23:10:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.017991+0000", 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025437464 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018087+0000", 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5035000000000001e-05 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018227+0000", 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140722 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35585 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992557+0000", 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056507137000001, 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025811557999999998, 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35585 2024-09-21T23:10:45.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992557+0000", 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992557+0000", 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992558+0000", 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.05e-07 2024-09-21T23:10:45.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992559+0000", 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0389999999999999e-06 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992562+0000", 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4149999999999999e-06 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992564+0000", 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.491e-06 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018243+0000", 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025678326000000001 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018291+0000", 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8028999999999997e-05 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018369+0000", 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8053000000000006e-05 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35586 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 2588672~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992568+0000", 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056496320999999, 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026111342999999999, 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35586 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.106 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992568+0000", 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992568+0000", 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992568+0000", 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992577+0000", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992581+0000", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.391e-06 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992584+0000", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7310000000000002e-06 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018402+0000", 2024-09-21T23:10:45.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025818358 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018514+0000", 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011228 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018679+0000", 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165014 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35587 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 2850816~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992587+0000", 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056477111, 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026224998999999999, 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35587 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992587+0000", 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992587+0000", 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992588+0000", 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.29e-07 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992589+0000", 2024-09-21T23:10:45.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1519999999999999e-06 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992592+0000", 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5250000000000001e-06 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992595+0000", 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3750000000000001e-06 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018693+0000", 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026098541999999999 2024-09-21T23:10:45.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018738+0000", 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4453999999999998e-05 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018812+0000", 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4721999999999994e-05 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35588 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992598+0000", 2024-09-21T23:10:45.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056466277, 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026518697000000001, 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35588 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992598+0000", 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992598+0000", 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992598+0000", 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4999999999999999e-07 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992599+0000", 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992604+0000", 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.938e-06 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992609+0000", 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.656e-06 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018847+0000", 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026237963 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.018960+0000", 2024-09-21T23:10:45.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000112664 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019117+0000", 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000157193 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35589 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 3375104~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992628+0000", 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056436395, 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026680764999999999, 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35589 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992628+0000", 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992628+0000", 2024-09-21T23:10:45.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992629+0000", 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992631+0000", 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.097e-06 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992637+0000", 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.127e-06 2024-09-21T23:10:45.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992643+0000", 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0990000000000004e-06 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019165+0000", 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026521797 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019225+0000", 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0470999999999998e-05 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019309+0000", 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3658000000000004e-05 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35590 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992650+0000", 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056414815, 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026960873, 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35590 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992650+0000", 2024-09-21T23:10:45.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992650+0000", 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992650+0000", 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6399999999999998e-07 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992652+0000", 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9319999999999999e-06 2024-09-21T23:10:45.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992661+0000", 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6079999999999996e-06 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992676+0000", 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4976e-05 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019350+0000", 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026674939000000002 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019444+0000", 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3805000000000001e-05 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019611+0000", 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016624899999999999 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35591 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992683+0000", 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056381291999999, 2024-09-21T23:10:45.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02710164, 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35591 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992683+0000", 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992683+0000", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992684+0000", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992686+0000", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2350000000000002e-06 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992694+0000", 2024-09-21T23:10:45.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0679999999999997e-06 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992699+0000", 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9089999999999999e-06 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019650+0000", 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026951559 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019708+0000", 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7626e-05 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019785+0000", 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6824999999999997e-05 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:35592 3.1a 3:5ccbb74a:::rbd_data.1149a5588948.0000000000000005:16 [read 4161536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:33.992705+0000", 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.056359684, 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.027375357999999999, 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:45.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 35592 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992705+0000", 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992705+0000", 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:45.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992705+0000", 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992707+0000", 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992716+0000", 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3130000000000004e-06 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:33.992722+0000", 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0679999999999996e-06 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019826+0000", 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.027104572 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.019929+0000", 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102877 2024-09-21T23:10:45.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:34.020080+0000", 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015111899999999999 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:45.133 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:45.134 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:45.135 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:45.136+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:45.235 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:45.232+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:45.253 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:45.253 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:45.254 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:45.254 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:45.267 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:45.336 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:45.336+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:45.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:45.435 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:45.435 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:45.438 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:45.436+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:45.450 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:45.493 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:45.493 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:45.509 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:45.510 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:45.526 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:45.526 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:45.538 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:45.536+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:45.544 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:45.544 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:45.544 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:45.565 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:45.565 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:45.601 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:45.602 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:45.632 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:45.632 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:45.637 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:45.637 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:45.639 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:45.640+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:45.641 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:45.642 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:45.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:45.740 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:45.740+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:45.746 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:10:45.818 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:45.818 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:45.834 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:45.840 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:45.840+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:45.859 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone23 from 0xe6efff7 (overlap 0x18ce8c6) to 0x7712a1 2024-09-21T23:10:45.941 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:45.940+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:45.983 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:45.983 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:46.000 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:46.041 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:46.040+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:46.071 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-21T23:10:46.143 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:46.140+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:46.172 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:46.173 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:46.188 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:46.244 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:46.244+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:46.287 INFO:teuthology.orchestra.run.smithi150.stdout:2771 write 0x90306da thru 0x90385b4 (0x7edb bytes) 2024-09-21T23:10:46.300 INFO:teuthology.orchestra.run.smithi150.stdout:2773 write 0x7f95e4b thru 0x7f96994 (0xb4a bytes) 2024-09-21T23:10:46.307 INFO:teuthology.orchestra.run.smithi150.stdout:2774 writesame 0x57c8d8c thru 0x57ceaed data_size 0x1(0x5d62 bytes) 2024-09-21T23:10:46.319 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:46.319 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:46.334 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:46.345 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:46.344+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:46.352 INFO:teuthology.orchestra.run.smithi150.stdout:2775 trunc from 0xe6efff7 to 0x6156a25 2024-09-21T23:10:46.392 INFO:teuthology.orchestra.run.smithi150.stdout:2776 read 0x5d90d96 thru 0x5d92ddb (0x2046 bytes) 2024-09-21T23:10:46.393 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:46.393 INFO:teuthology.orchestra.run.smithi150.stdout:2779 clone 25 order 21 su 32768 sc 3 2024-09-21T23:10:46.446 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:46.444+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:46.499 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:46.499 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:46.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:46.547 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:46.544+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:46.631 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:46.631 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:46.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:46.647 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:46.647+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:46.749 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:46.748+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:46.756 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T23:10:46.756 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:10:46.756 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T23:10:46.756 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:10:46.769 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:46.769 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:46.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:46.849 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:46.847+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:46.907 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:46.907 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:46.913 INFO:teuthology.orchestra.run.smithi150.stdout:leaving image image_client.0-clone24 intact 2024-09-21T23:10:46.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:46.950 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:46.948+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:47.051 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:47.051+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:47.087 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:10:47.087 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:47.087 INFO:teuthology.orchestra.run.smithi150.stdout: "epoch": 58, 2024-09-21T23:10:47.087 INFO:teuthology.orchestra.run.smithi150.stdout: "fsid": "9ce7cfc1-a7d6-426f-bf44-a308fb883b8f", 2024-09-21T23:10:47.087 INFO:teuthology.orchestra.run.smithi150.stdout: "created": "2024-09-21T23:09:25.662940+0000", 2024-09-21T23:10:47.087 INFO:teuthology.orchestra.run.smithi150.stdout: "modified": "2024-09-21T23:10:46.758331+0000", 2024-09-21T23:10:47.087 INFO:teuthology.orchestra.run.smithi150.stdout: "last_up_change": "2024-09-21T23:10:08.438076+0000", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "last_in_change": "2024-09-21T23:10:35.668168+0000", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_num": 5806080, 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_set": [ 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "nodeep-scrub", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "noscrub", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "pglog_hardlimit", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "purged_snapdirs", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_deletes", 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "sortbitwise" 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_version": 4, 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:10:47.088 INFO:teuthology.orchestra.run.smithi150.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_snapshot": "", 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_max": 3, 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd": 8, 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "min_compat_client": "jewel", 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "require_osd_release": "squid", 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_crimson": false, 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "pools": [ 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 1, 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": ".mgr", 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:40.569134+0000", 2024-09-21T23:10:47.089 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool", 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:10:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 1, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 1, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 1, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 1, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 1, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "16", 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:10:47.091 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 0, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 0, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:10:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:10:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "options": { 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_max": 32, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_min": 1 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.094 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": {} 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 8, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 8, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 0.25, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 2, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 2, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 2, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": "rbd", 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:44.626156+0000", 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 8193, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:10:47.096 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 8, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 8, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 8, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 8, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 8, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:10:47.097 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "20", 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 2, 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 20, 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:10:47.098 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:10:47.099 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:10:47.100 INFO:teuthology.orchestra.run.smithi150.stdout: "options": {}, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd": {} 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 3, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 3, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 0.75, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 2.25, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 2.25, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:10:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 3, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": "pool_client.0", 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:53.888601+0000", 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 8193, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:10:47.102 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 32, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 32, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 32, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 32, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 32, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:10:47.103 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "58", 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 27, 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 58, 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:10:47.104 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:10:47.105 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:10:47.106 INFO:teuthology.orchestra.run.smithi150.stdout: "options": {}, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd": {} 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 2, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 2, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 0.75, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 1.5, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 1.5, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:10:47.107 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 0, 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d", 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 53, 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6816", 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6817", 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6818", 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6819", 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.110 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6822", 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6823", 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.111 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6820", 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6821", 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:10:47.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6817/198356244", 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6819/198356244", 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6823/198356244", 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6821/198356244", 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 1, 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "5fb69564-3b15-470e-b45f-f70594751344", 2024-09-21T23:10:47.113 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 53, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.114 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6824", 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6825", 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6826", 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6827", 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6830", 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6831", 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.117 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6828", 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6829", 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6825/725319294", 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6827/725319294", 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6831/725319294", 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6829/725319294", 2024-09-21T23:10:47.118 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 2, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "cc3e3e88-a324-4e3d-8b40-0d8c93da1400", 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.119 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 53, 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6800", 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6801", 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.120 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6802", 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6803", 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6806", 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6807", 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.122 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6804", 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6805", 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6801/1956153436", 2024-09-21T23:10:47.123 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6803/1956153436", 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6807/1956153436", 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6805/1956153436", 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 3, 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "933fe042-8aed-4795-918c-572cf81999a7", 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.124 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 53, 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6808", 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6809", 2024-09-21T23:10:47.125 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6810", 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6811", 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6814", 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6815", 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.127 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6812", 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6813", 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6809/3049870482", 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6811/3049870482", 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6815/3049870482", 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6813/3049870482", 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 4, 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "af826961-2dad-4702-a904-991e2f5f57c8", 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 0, 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:10:47.129 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 21, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 33, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6800", 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.130 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.131 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6801", 2024-09-21T23:10:47.131 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.131 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.131 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.131 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6802", 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6803", 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6806", 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6807", 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6804", 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6805", 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 4112998952 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6801/4112998952", 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6803/4112998952", 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6807/4112998952", 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6805/4112998952", 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "exists" 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 5, 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "c4f1908b-0885-408e-873e-4d2aa9b9c506", 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:10:47.137 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 53, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6826", 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6827", 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6828", 2024-09-21T23:10:47.139 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6829", 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6832", 2024-09-21T23:10:47.140 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6833", 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6830", 2024-09-21T23:10:47.141 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6831", 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6827/120048378", 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6829/120048378", 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6833/120048378", 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6831/120048378", 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.142 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 6, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "08cef493-ce38-4b52-b5f2-23e7694953c0", 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.143 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 53, 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6818", 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6819", 2024-09-21T23:10:47.144 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6820", 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6821", 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6824", 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6825", 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.146 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6822", 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6823", 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.147 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6819/787648514", 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6821/787648514", 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6825/787648514", 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6823/787648514", 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 7, 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "26d2142e-419c-4da3-9c28-ac442b1b66fb", 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 0, 2024-09-21T23:10:47.148 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 0, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 52, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6808", 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6809", 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6810", 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.150 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6811", 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6814", 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6815", 2024-09-21T23:10:47.151 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6812", 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6813", 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:10:47.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6809/1424462696", 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6811/1424462696", 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6815/1424462696", 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6813/1424462696", 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_xinfo": [ 2024-09-21T23:10:47.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 0, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:36.004700+0000", 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 1, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.154 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.958497+0000", 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 2, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.988302+0000", 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:10:47.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 3, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.993971+0000", 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 4, 2024-09-21T23:10:47.156 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.945648+0000", 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 33 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 5, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:10:47.157 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:36.366302+0000", 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 6, 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.852613+0000", 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.158 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 7, 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 65536, 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.713697+0000", 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap": [], 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap_items": [], 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap_primaries": [], 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_temp": [ 2024-09-21T23:10:47.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "2.5", 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: 0, 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: 7 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "3.e", 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: 1, 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: 7 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "3.f", 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: 0, 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: 7 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "3.11", 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: 6, 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: 7 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.161 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "3.13", 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: 2, 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: 7 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "3.14", 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: 6, 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: 7 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "3.15", 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: 3, 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: 7 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_temp": [], 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklist": {}, 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "range_blocklist": {}, 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profiles": { 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "default": { 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-failure-domain": "osd", 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "k": "2", 2024-09-21T23:10:47.163 INFO:teuthology.orchestra.run.smithi150.stdout: "m": "1", 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "plugin": "jerasure", 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "technique": "reed_sol_van" 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps_queue": [], 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "new_removed_snaps": [], 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "new_purged_snaps": [], 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_node_flags": {}, 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "device_class_flags": {}, 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode": { 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode_enabled": false, 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_bucket_count": 0, 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:10:47.164 INFO:teuthology.orchestra.run.smithi150.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:10:47.165 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode_bucket": 0 2024-09-21T23:10:47.165 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:47.165 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:47.167 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:47.167 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:47.167 INFO:tasks.thrashosds.thrasher:j is {'epoch': 58, 'fsid': '9ce7cfc1-a7d6-426f-bf44-a308fb883b8f', 'created': '2024-09-21T23:09:25.662940+0000', 'modified': '2024-09-21T23:10:46.758331+0000', 'last_up_change': '2024-09-21T23:10:08.438076+0000', 'last_in_change': '2024-09-21T23:10:35.668168+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:09:40.569134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:09:44.626156+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T23:09:53.888601+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 58, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 198356244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 198356244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 198356244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 198356244}]}, 'public_addr': '172.21.15.150:6817/198356244', 'cluster_addr': '172.21.15.150:6819/198356244', 'heartbeat_back_addr': '172.21.15.150:6823/198356244', 'heartbeat_front_addr': '172.21.15.150:6821/198356244', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5fb69564-3b15-470e-b45f-f70594751344', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 725319294}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 725319294}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 725319294}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 725319294}]}, 'public_addr': '172.21.15.150:6825/725319294', 'cluster_addr': '172.21.15.150:6827/725319294', 'heartbeat_back_addr': '172.21.15.150:6831/725319294', 'heartbeat_front_addr': '172.21.15.150:6829/725319294', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'cc3e3e88-a324-4e3d-8b40-0d8c93da1400', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 1956153436}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 1956153436}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 1956153436}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 1956153436}]}, 'public_addr': '172.21.15.150:6801/1956153436', 'cluster_addr': '172.21.15.150:6803/1956153436', 'heartbeat_back_addr': '172.21.15.150:6807/1956153436', 'heartbeat_front_addr': '172.21.15.150:6805/1956153436', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '933fe042-8aed-4795-918c-572cf81999a7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 3049870482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 3049870482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 3049870482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 3049870482}]}, 'public_addr': '172.21.15.150:6809/3049870482', 'cluster_addr': '172.21.15.150:6811/3049870482', 'heartbeat_back_addr': '172.21.15.150:6815/3049870482', 'heartbeat_front_addr': '172.21.15.150:6813/3049870482', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'af826961-2dad-4702-a904-991e2f5f57c8', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 33, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6801', 'nonce': 4112998952}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6803', 'nonce': 4112998952}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6807', 'nonce': 4112998952}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 4112998952}, {'type': 'v1', 'addr': '172.21.15.175:6805', 'nonce': 4112998952}]}, 'public_addr': '172.21.15.175:6801/4112998952', 'cluster_addr': '172.21.15.175:6803/4112998952', 'heartbeat_back_addr': '172.21.15.175:6807/4112998952', 'heartbeat_front_addr': '172.21.15.175:6805/4112998952', 'state': ['exists']}, {'osd': 5, 'uuid': 'c4f1908b-0885-408e-873e-4d2aa9b9c506', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6826', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6827', 'nonce': 120048378}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6828', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6829', 'nonce': 120048378}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 120048378}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6830', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6831', 'nonce': 120048378}]}, 'public_addr': '172.21.15.175:6827/120048378', 'cluster_addr': '172.21.15.175:6829/120048378', 'heartbeat_back_addr': '172.21.15.175:6833/120048378', 'heartbeat_front_addr': '172.21.15.175:6831/120048378', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '08cef493-ce38-4b52-b5f2-23e7694953c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6818', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6819', 'nonce': 787648514}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6820', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6821', 'nonce': 787648514}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6824', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6825', 'nonce': 787648514}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6822', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6823', 'nonce': 787648514}]}, 'public_addr': '172.21.15.175:6819/787648514', 'cluster_addr': '172.21.15.175:6821/787648514', 'heartbeat_back_addr': '172.21.15.175:6825/787648514', 'heartbeat_front_addr': '172.21.15.175:6823/787648514', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '26d2142e-419c-4da3-9c28-ac442b1b66fb', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6809', 'nonce': 1424462696}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6811', 'nonce': 1424462696}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6815', 'nonce': 1424462696}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6813', 'nonce': 1424462696}]}, 'public_addr': '172.21.15.175:6809/1424462696', 'cluster_addr': '172.21.15.175:6811/1424462696', 'heartbeat_back_addr': '172.21.15.175:6815/1424462696', 'heartbeat_front_addr': '172.21.15.175:6813/1424462696', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:36.004700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.958497+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.988302+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.993971+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.945648+0000', 'dead_epoch': 33}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:36.366302+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.852613+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.713697+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '2.5', 'osds': [0, 7]}, {'pgid': '3.e', 'osds': [1, 7]}, {'pgid': '3.f', 'osds': [0, 7]}, {'pgid': '3.11', 'osds': [6, 7]}, {'pgid': '3.13', 'osds': [2, 7]}, {'pgid': '3.14', 'osds': [6, 7]}, {'pgid': '3.15', 'osds': [3, 7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:10:47.168 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:10:47.168 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-21T23:10:47.168 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:47.169 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:47.163+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:47.170 INFO:teuthology.orchestra.run.smithi150.stdout:2782 write 0x7b5192c thru 0x7b53233 (0x1908 bytes) 2024-09-21T23:10:47.174 INFO:teuthology.orchestra.run.smithi150.stdout:2783 writesame 0x39b6542 thru 0x39bc0c7 data_size 0x1(0x5b86 bytes) 2024-09-21T23:10:47.190 INFO:teuthology.orchestra.run.smithi150.stdout:2784 write 0x725fd2d thru 0x7260c57 (0xf2b bytes) 2024-09-21T23:10:47.196 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:47.208 INFO:teuthology.orchestra.run.smithi150.stdout:2788 write 0xb6ddfb4 thru 0xb6e5668 (0x76b5 bytes) 2024-09-21T23:10:47.213 INFO:teuthology.orchestra.run.smithi150.stdout:2789 read 0x303bd93 thru 0x3047405 (0xb673 bytes) 2024-09-21T23:10:47.213 INFO:teuthology.orchestra.run.smithi150.stdout:2790 read 0xb08d1c4 thru 0xb092a35 (0x5872 bytes) 2024-09-21T23:10:47.213 INFO:teuthology.orchestra.run.smithi150.stdout:2791 writesame 0xa1177f3 thru 0xa1195e1 data_size 0x1(0x1def bytes) 2024-09-21T23:10:47.220 INFO:teuthology.orchestra.run.smithi150.stdout:2794 read 0x2516705 thru 0x2524dde (0xe6da bytes) 2024-09-21T23:10:47.220 INFO:teuthology.orchestra.run.smithi150.stdout:2795 read 0x59afe6a thru 0x59b2483 (0x261a bytes) 2024-09-21T23:10:47.227 INFO:teuthology.orchestra.run.smithi150.stdout:2796 trunc from 0xb6e5669 to 0x6317d3b 2024-09-21T23:10:47.247 INFO:teuthology.orchestra.run.smithi150.stdout:2797 write 0x1b2e96d thru 0x1b39cab (0xb33f bytes) 2024-09-21T23:10:47.266 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:47.267+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:47.304 INFO:teuthology.orchestra.run.smithi150.stdout:2799 writesame 0x9f87d0f thru 0x9f90395 data_size 0x1(0x8687 bytes) 2024-09-21T23:10:47.305 INFO:teuthology.orchestra.run.smithi150.stdout:2800 write 0x168d32e thru 0x16956cd (0x83a0 bytes) 2024-09-21T23:10:47.351 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:47.351 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:47.362 INFO:teuthology.orchestra.run.smithi150.stdout:2801 write 0xd654c15 thru 0xd6565a7 (0x1993 bytes) 2024-09-21T23:10:47.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:47.366 INFO:teuthology.orchestra.run.smithi150.stdout:2802 read 0x957e204 thru 0x958496c (0x6769 bytes) 2024-09-21T23:10:47.367 INFO:teuthology.orchestra.run.smithi150.stdout:2804 read 0x39e26d5 thru 0x39f26d3 (0xffff bytes) 2024-09-21T23:10:47.367 INFO:teuthology.orchestra.run.smithi150.stdout:2805 write 0xd00d04e thru 0xd0176ca (0xa67d bytes) 2024-09-21T23:10:47.371 INFO:teuthology.orchestra.run.smithi150.stdout:2806 write 0x9bfa2de thru 0x9bff9e6 (0x5709 bytes) 2024-09-21T23:10:47.374 INFO:teuthology.orchestra.run.smithi150.stdout:2807 punch from 0x8b17b7d to 0x8b1d0f0, (0x5573 bytes) 2024-09-21T23:10:47.374 INFO:teuthology.orchestra.run.smithi150.stdout:2808 read 0xd46a154 thru 0xd46c352 (0x21ff bytes) 2024-09-21T23:10:47.375 INFO:teuthology.orchestra.run.smithi150.stdout:2809 punch from 0x5992528 to 0x599eea8, (0xc980 bytes) 2024-09-21T23:10:47.375 INFO:teuthology.orchestra.run.smithi150.stdout:2811 trunc from 0xd6565a8 to 0xcce952c 2024-09-21T23:10:47.388 INFO:teuthology.orchestra.run.smithi150.stdout:2813 writesame 0xac07c73 thru 0xac0ba6d data_size 0x2b(0x3dfb bytes) 2024-09-21T23:10:47.392 INFO:teuthology.orchestra.run.smithi150.stdout:2814 write 0xa8fb7e4 thru 0xa9030b7 (0x78d4 bytes) 2024-09-21T23:10:47.396 INFO:teuthology.orchestra.run.smithi150.stdout:2816 read 0x89effc7 thru 0x89f7826 (0x7860 bytes) 2024-09-21T23:10:47.396 INFO:teuthology.orchestra.run.smithi150.stdout:2817 read 0x47a26c8 thru 0x47aabdf (0x8518 bytes) 2024-09-21T23:10:47.404 INFO:teuthology.orchestra.run.smithi150.stdout:2818 write 0xed03886 thru 0xed0f922 (0xc09d bytes) 2024-09-21T23:10:47.407 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:47.407+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:47.409 INFO:teuthology.orchestra.run.smithi150.stdout:2819 writesame 0x76a02c7 thru 0x76a8703 data_size 0x1(0x843d bytes) 2024-09-21T23:10:47.428 INFO:teuthology.orchestra.run.smithi150.stdout:2820 write 0xdb5c1f5 thru 0xdb651ea (0x8ff6 bytes) 2024-09-21T23:10:47.432 INFO:teuthology.orchestra.run.smithi150.stdout:2821 writesame 0x7a0c973 thru 0x7a10f27 data_size 0x2b(0x45b5 bytes) 2024-09-21T23:10:47.436 INFO:teuthology.orchestra.run.smithi150.stdout:2822 write 0x4249ca1 thru 0x425406c (0xa3cc bytes) 2024-09-21T23:10:47.468 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:47.468+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:47.475 INFO:teuthology.orchestra.run.smithi150.stdout:2823 trunc from 0xed0f923 to 0xb951a5f 2024-09-21T23:10:47.486 INFO:teuthology.orchestra.run.smithi150.stdout:2824 write 0x47628c3 thru 0x4764c42 (0x2380 bytes) 2024-09-21T23:10:47.495 INFO:teuthology.orchestra.run.smithi150.stdout:2826 trunc from 0xb951a5f to 0x7bcd1cf 2024-09-21T23:10:47.510 INFO:teuthology.orchestra.run.smithi150.stdout:2828 punch from 0x14e655a to 0x14e956c, (0x3012 bytes) 2024-09-21T23:10:47.510 INFO:teuthology.orchestra.run.smithi150.stdout:2829 read 0x757222e thru 0x757f4ed (0xd2c0 bytes) 2024-09-21T23:10:47.510 INFO:teuthology.orchestra.run.smithi150.stdout:2831 punch from 0x42282b2 to 0x423462a, (0xc378 bytes) 2024-09-21T23:10:47.510 INFO:teuthology.orchestra.run.smithi150.stdout:2832 write 0x4e65ba8 thru 0x4e7441c (0xe875 bytes) 2024-09-21T23:10:47.519 INFO:teuthology.orchestra.run.smithi150.stdout:2837 read 0x1586138 thru 0x1589b35 (0x39fe bytes) 2024-09-21T23:10:47.521 INFO:teuthology.orchestra.run.smithi150.stdout:2838 write 0x2d07ccb thru 0x2d15884 (0xdbba bytes) 2024-09-21T23:10:47.530 INFO:teuthology.orchestra.run.smithi150.stdout:2839 write 0x629530b thru 0x629c839 (0x752f bytes) 2024-09-21T23:10:47.532 INFO:teuthology.orchestra.run.smithi150.stdout:2840 writesame 0x69275b3 thru 0x692ff86 data_size 0x2(0x89d4 bytes) 2024-09-21T23:10:47.541 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:47.541 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:47.544 INFO:teuthology.orchestra.run.smithi150.stdout:2841 trunc from 0x7bcd1cf to 0x5c25482 2024-09-21T23:10:47.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:10:47.562 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:10:47.590 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-21T23:10:46.768496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":994168829,"num_objects":1333,"num_object_clones":203,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":330,"num_objects_misplaced":252,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":720,"num_read":62460,"num_read_kb":446220,"num_write":9449,"num_write_kb":234994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":59967499,"num_keys_recovered":47,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7943,"ondisk_log_size":7943,"up":67,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":538660,"kb_used_data":303508,"kb_used_omap":10,"kb_used_meta":234805,"kb_avail":660064220,"statfs":{"total":676457349120,"available":675905761280,"internally_reserved":0,"allocated":310792192,"data_stored":331013252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11160,"internal_metadata":240440424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":146104536,"num_objects":113,"num_object_clones":5,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":19,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":14,"num_read":8250,"num_read_kb":47065,"num_write":1056,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.034030"},"pg_stats":[{"pgid":"3.1f","version":"56'154","reported_seq":1399,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.726423+0000","last_change":"2024-09-21T23:09:57.384622+0000","last_active":"2024-09-21T23:10:44.726423+0000","last_peered":"2024-09-21T23:10:44.726423+0000","last_clean":"2024-09-21T23:10:44.726423+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:10:44.726423+0000","last_undegraded":"2024-09-21T23:10:44.726423+0000","last_fullsized":"2024-09-21T23:10:44.726423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:43:26.380005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018862200000000001,"stat_sum":{"num_bytes":24365758,"num_objects":36,"num_object_clones":4,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":1235,"num_read_kb":9062,"num_write":138,"num_write_kb":4072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"56'317","reported_seq":1746,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.802316+0000","last_change":"2024-09-21T23:09:57.368121+0000","last_active":"2024-09-21T23:10:44.802316+0000","last_peered":"2024-09-21T23:10:44.802316+0000","last_clean":"2024-09-21T23:10:44.802316+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:10:44.802316+0000","last_undegraded":"2024-09-21T23:10:44.802316+0000","last_fullsized":"2024-09-21T23:10:44.802316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:17.455022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010293570000000001,"stat_sum":{"num_bytes":35197553,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":2391,"num_read_kb":22033,"num_write":411,"num_write_kb":11546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"56'176","reported_seq":995,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.721894+0000","last_change":"2024-09-21T23:10:09.471685+0000","last_active":"2024-09-21T23:10:44.721894+0000","last_peered":"2024-09-21T23:10:44.721894+0000","last_clean":"2024-09-21T23:10:07.611778+0000","last_became_active":"2024-09-21T23:10:09.471685+0000","last_became_peered":"2024-09-21T23:10:09.471685+0000","last_unstale":"2024-09-21T23:10:44.721894+0000","last_undegraded":"2024-09-21T23:10:09.470679+0000","last_fullsized":"2024-09-21T23:10:09.470596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:02.270929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057415099999999996,"stat_sum":{"num_bytes":31275005,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":1599,"num_read_kb":6085,"num_write":165,"num_write_kb":5396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":34}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"56'177","reported_seq":1798,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.473961+0000","last_change":"2024-09-21T23:10:09.473355+0000","last_active":"2024-09-21T23:10:44.473961+0000","last_peered":"2024-09-21T23:10:44.473961+0000","last_clean":"2024-09-21T23:10:07.923091+0000","last_became_active":"2024-09-21T23:10:09.473355+0000","last_became_peered":"2024-09-21T23:10:09.473355+0000","last_unstale":"2024-09-21T23:10:44.473961+0000","last_undegraded":"2024-09-21T23:10:09.470552+0000","last_fullsized":"2024-09-21T23:10:09.470443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:39.094901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090318899999999999,"stat_sum":{"num_bytes":24540312,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":1832,"num_read_kb":9527,"num_write":174,"num_write_kb":6849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":38}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'255","reported_seq":2667,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.720641+0000","last_change":"2024-09-21T23:10:09.468036+0000","last_active":"2024-09-21T23:10:44.720641+0000","last_peered":"2024-09-21T23:10:44.720641+0000","last_clean":"2024-09-21T23:10:07.735659+0000","last_became_active":"2024-09-21T23:10:09.468036+0000","last_became_peered":"2024-09-21T23:10:09.468036+0000","last_unstale":"2024-09-21T23:10:44.720641+0000","last_undegraded":"2024-09-21T23:10:09.466428+0000","last_fullsized":"2024-09-21T23:10:09.466306+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:02:39.763091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007694,"stat_sum":{"num_bytes":19534013,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":28,"num_read":3819,"num_read_kb":15181,"num_write":332,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":48}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"56'608","reported_seq":2894,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.929074+0000","last_change":"2024-09-21T23:10:09.466761+0000","last_active":"2024-09-21T23:10:44.929074+0000","last_peered":"2024-09-21T23:10:44.929074+0000","last_clean":"2024-09-21T23:10:07.890822+0000","last_became_active":"2024-09-21T23:10:09.466761+0000","last_became_peered":"2024-09-21T23:10:09.466761+0000","last_unstale":"2024-09-21T23:10:44.929074+0000","last_undegraded":"2024-09-21T23:10:09.465786+0000","last_fullsized":"2024-09-21T23:10:09.465702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:33.884007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011201469999999999,"stat_sum":{"num_bytes":25155164,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":5762,"num_read_kb":18932,"num_write":880,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"56'720","reported_seq":1032,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.238684+0000","last_change":"2024-09-21T23:09:57.374916+0000","last_active":"2024-09-21T23:10:44.238684+0000","last_peered":"2024-09-21T23:10:44.238684+0000","last_clean":"2024-09-21T23:10:44.238684+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:10:44.238684+0000","last_undegraded":"2024-09-21T23:10:44.238684+0000","last_fullsized":"2024-09-21T23:10:44.238684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:36.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073844600000000002,"stat_sum":{"num_bytes":14440691,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":3602,"num_read_kb":12089,"num_write":1229,"num_write_kb":6003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"56'194","reported_seq":1402,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.826991+0000","last_change":"2024-09-21T23:09:57.367436+0000","last_active":"2024-09-21T23:10:44.826991+0000","last_peered":"2024-09-21T23:10:44.826991+0000","last_clean":"2024-09-21T23:10:44.826991+0000","last_became_active":"2024-09-21T23:09:55.366388+0000","last_became_peered":"2024-09-21T23:09:55.366388+0000","last_unstale":"2024-09-21T23:10:44.826991+0000","last_undegraded":"2024-09-21T23:10:44.826991+0000","last_fullsized":"2024-09-21T23:10:44.826991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:49:49.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054973999999999995,"stat_sum":{"num_bytes":27334427,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":1507,"num_read_kb":7299,"num_write":231,"num_write_kb":3661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'205","reported_seq":2030,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.897359+0000","last_change":"2024-09-21T23:09:57.384681+0000","last_active":"2024-09-21T23:10:44.897359+0000","last_peered":"2024-09-21T23:10:44.897359+0000","last_clean":"2024-09-21T23:10:44.897359+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:10:44.897359+0000","last_undegraded":"2024-09-21T23:10:44.897359+0000","last_fullsized":"2024-09-21T23:10:44.897359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:24:45.923904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001819836,"stat_sum":{"num_bytes":48619136,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":28,"num_read":1878,"num_read_kb":12497,"num_write":193,"num_write_kb":9077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"56'153","reported_seq":1633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.887591+0000","last_change":"2024-09-21T23:10:39.723527+0000","last_active":"2024-09-21T23:10:44.887591+0000","last_peered":"2024-09-21T23:10:44.887591+0000","last_clean":"2024-09-21T23:10:44.887591+0000","last_became_active":"2024-09-21T23:10:39.722187+0000","last_became_peered":"2024-09-21T23:10:39.722187+0000","last_unstale":"2024-09-21T23:10:44.887591+0000","last_undegraded":"2024-09-21T23:10:44.887591+0000","last_fullsized":"2024-09-21T23:10:44.887591+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:11:50.105159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049884500000000004,"stat_sum":{"num_bytes":35781382,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":1871,"num_read_kb":15014,"num_write":128,"num_write_kb":6286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5583924,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"56'220","reported_seq":1755,"reported_epoch":56,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:10:44.836251+0000","last_change":"2024-09-21T23:10:37.687287+0000","last_active":"2024-09-21T23:10:44.836251+0000","last_peered":"2024-09-21T23:10:44.836251+0000","last_clean":"2024-09-21T23:10:44.836251+0000","last_became_active":"2024-09-21T23:10:37.686468+0000","last_became_peered":"2024-09-21T23:10:37.686468+0000","last_unstale":"2024-09-21T23:10:44.836251+0000","last_undegraded":"2024-09-21T23:10:44.836251+0000","last_fullsized":"2024-09-21T23:10:44.836251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:43.916106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313919,"stat_sum":{"num_bytes":60397154,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":2594,"num_read_kb":18034,"num_write":206,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[{"shards":"3,7","objects":39}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"56'317","reported_seq":1281,"reported_epoch":56,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:10:44.783039+0000","last_change":"2024-09-21T23:10:39.719086+0000","last_active":"2024-09-21T23:10:44.783039+0000","last_peered":"2024-09-21T23:10:44.783039+0000","last_clean":"2024-09-21T23:10:44.783039+0000","last_became_active":"2024-09-21T23:10:39.717785+0000","last_became_peered":"2024-09-21T23:10:39.717785+0000","last_unstale":"2024-09-21T23:10:44.783039+0000","last_undegraded":"2024-09-21T23:10:44.783039+0000","last_fullsized":"2024-09-21T23:10:44.783039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:52:15.713179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110800000000001,"stat_sum":{"num_bytes":12681967,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":2044,"num_read_kb":5355,"num_write":357,"num_write_kb":2630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1527429,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":39}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.817263+0000","last_change":"2024-09-21T23:10:36.693642+0000","last_active":"2024-09-21T23:10:42.817263+0000","last_peered":"2024-09-21T23:10:42.817263+0000","last_clean":"2024-09-21T23:10:42.817263+0000","last_became_active":"2024-09-21T23:10:36.692757+0000","last_became_peered":"2024-09-21T23:10:36.692757+0000","last_unstale":"2024-09-21T23:10:42.817263+0000","last_undegraded":"2024-09-21T23:10:42.817263+0000","last_fullsized":"2024-09-21T23:10:42.817263+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:02.633397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"56'128","reported_seq":468,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.863155+0000","last_change":"2024-09-21T23:09:57.383456+0000","last_active":"2024-09-21T23:10:44.863155+0000","last_peered":"2024-09-21T23:10:44.863155+0000","last_clean":"2024-09-21T23:10:44.863155+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:10:44.863155+0000","last_undegraded":"2024-09-21T23:10:44.863155+0000","last_fullsized":"2024-09-21T23:10:44.863155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:48.970048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069300299999999998,"stat_sum":{"num_bytes":51029283,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":275,"num_read_kb":13241,"num_write":122,"num_write_kb":7920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.758775+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:10:42.758775+0000","last_peered":"2024-09-21T23:10:42.758775+0000","last_clean":"2024-09-21T23:10:42.758775+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:10:42.758775+0000","last_undegraded":"2024-09-21T23:10:42.758775+0000","last_fullsized":"2024-09-21T23:10:42.758775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"56'229","reported_seq":804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.577430+0000","last_change":"2024-09-21T23:10:39.718517+0000","last_active":"2024-09-21T23:10:44.577430+0000","last_peered":"2024-09-21T23:10:44.577430+0000","last_clean":"2024-09-21T23:10:44.577430+0000","last_became_active":"2024-09-21T23:10:39.717971+0000","last_became_peered":"2024-09-21T23:10:39.717971+0000","last_unstale":"2024-09-21T23:10:44.577430+0000","last_undegraded":"2024-09-21T23:10:44.577430+0000","last_fullsized":"2024-09-21T23:10:44.577430+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:57.120189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333359999999999,"stat_sum":{"num_bytes":59845237,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":962,"num_read_kb":16785,"num_write":297,"num_write_kb":14901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":11539207,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":56,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:10:42.751196+0000","last_change":"2024-09-21T23:10:37.691358+0000","last_active":"2024-09-21T23:10:42.751196+0000","last_peered":"2024-09-21T23:10:42.751196+0000","last_clean":"2024-09-21T23:10:42.751196+0000","last_became_active":"2024-09-21T23:10:37.690925+0000","last_became_peered":"2024-09-21T23:10:37.690925+0000","last_unstale":"2024-09-21T23:10:42.751196+0000","last_undegraded":"2024-09-21T23:10:42.751196+0000","last_fullsized":"2024-09-21T23:10:42.751196+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:54.909632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"56'173","reported_seq":835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.869013+0000","last_change":"2024-09-21T23:09:57.375144+0000","last_active":"2024-09-21T23:10:44.869013+0000","last_peered":"2024-09-21T23:10:44.869013+0000","last_clean":"2024-09-21T23:10:44.869013+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:10:44.869013+0000","last_undegraded":"2024-09-21T23:10:44.869013+0000","last_fullsized":"2024-09-21T23:10:44.869013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:13:38.814637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079926499999999996,"stat_sum":{"num_bytes":21882219,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":601,"num_read_kb":10518,"num_write":154,"num_write_kb":8143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.759077+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:10:42.759077+0000","last_peered":"2024-09-21T23:10:42.759077+0000","last_clean":"2024-09-21T23:10:42.759077+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:10:42.759077+0000","last_undegraded":"2024-09-21T23:10:42.759077+0000","last_fullsized":"2024-09-21T23:10:42.759077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"56'415","reported_seq":2803,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.908668+0000","last_change":"2024-09-21T23:09:57.378889+0000","last_active":"2024-09-21T23:10:44.908668+0000","last_peered":"2024-09-21T23:10:44.908668+0000","last_clean":"2024-09-21T23:10:44.908668+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:10:44.908668+0000","last_undegraded":"2024-09-21T23:10:44.908668+0000","last_fullsized":"2024-09-21T23:10:44.908668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:29:17.925525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000364898,"stat_sum":{"num_bytes":60691593,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":3661,"num_read_kb":24896,"num_write":677,"num_write_kb":11028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"56'294","reported_seq":2116,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.802812+0000","last_change":"2024-09-21T23:10:09.467744+0000","last_active":"2024-09-21T23:10:44.802812+0000","last_peered":"2024-09-21T23:10:44.802812+0000","last_clean":"2024-09-21T23:10:07.635182+0000","last_became_active":"2024-09-21T23:10:09.467744+0000","last_became_peered":"2024-09-21T23:10:09.467744+0000","last_unstale":"2024-09-21T23:10:44.802812+0000","last_undegraded":"2024-09-21T23:10:09.466288+0000","last_fullsized":"2024-09-21T23:10:09.466187+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:43.410221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060347799999999996,"stat_sum":{"num_bytes":27460743,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":2724,"num_read_kb":17220,"num_write":353,"num_write_kb":6739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":38}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.752492+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:10:42.752492+0000","last_peered":"2024-09-21T23:10:42.752492+0000","last_clean":"2024-09-21T23:10:42.752492+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:10:42.752492+0000","last_undegraded":"2024-09-21T23:10:42.752492+0000","last_fullsized":"2024-09-21T23:10:42.752492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"56'136","reported_seq":806,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.871625+0000","last_change":"2024-09-21T23:09:57.374883+0000","last_active":"2024-09-21T23:10:44.871625+0000","last_peered":"2024-09-21T23:10:44.871625+0000","last_clean":"2024-09-21T23:10:44.871625+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:10:44.871625+0000","last_undegraded":"2024-09-21T23:10:44.871625+0000","last_fullsized":"2024-09-21T23:10:44.871625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:10.138100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060209499999999997,"stat_sum":{"num_bytes":22733267,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":654,"num_read_kb":10602,"num_write":131,"num_write_kb":8435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.799453+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:10:42.799453+0000","last_peered":"2024-09-21T23:10:42.799453+0000","last_clean":"2024-09-21T23:10:42.799453+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:10:42.799453+0000","last_undegraded":"2024-09-21T23:10:42.799453+0000","last_fullsized":"2024-09-21T23:10:42.799453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"56'146","reported_seq":1973,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.812178+0000","last_change":"2024-09-21T23:10:09.466917+0000","last_active":"2024-09-21T23:10:44.812178+0000","last_peered":"2024-09-21T23:10:44.812178+0000","last_clean":"2024-09-21T23:10:07.598263+0000","last_became_active":"2024-09-21T23:10:09.466917+0000","last_became_peered":"2024-09-21T23:10:09.466917+0000","last_unstale":"2024-09-21T23:10:44.812178+0000","last_undegraded":"2024-09-21T23:10:09.466191+0000","last_fullsized":"2024-09-21T23:10:09.466071+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:41.459765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021610209999999999,"stat_sum":{"num_bytes":29530647,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":1808,"num_read_kb":10552,"num_write":156,"num_write_kb":7108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.758393+0000","last_change":"2024-09-21T23:10:36.686691+0000","last_active":"2024-09-21T23:10:42.758393+0000","last_peered":"2024-09-21T23:10:42.758393+0000","last_clean":"2024-09-21T23:10:42.758393+0000","last_became_active":"2024-09-21T23:10:36.686287+0000","last_became_peered":"2024-09-21T23:10:36.686287+0000","last_unstale":"2024-09-21T23:10:42.758393+0000","last_undegraded":"2024-09-21T23:10:42.758393+0000","last_fullsized":"2024-09-21T23:10:42.758393+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:20.016368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"56'237","reported_seq":612,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.727312+0000","last_change":"2024-09-21T23:09:57.368695+0000","last_active":"2024-09-21T23:10:44.727312+0000","last_peered":"2024-09-21T23:10:44.727312+0000","last_clean":"2024-09-21T23:10:44.727312+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:10:44.727312+0000","last_undegraded":"2024-09-21T23:10:44.727312+0000","last_fullsized":"2024-09-21T23:10:44.727312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:38:35.363188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013976220000000001,"stat_sum":{"num_bytes":6316963,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":704,"num_read_kb":4765,"num_write":283,"num_write_kb":7751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.752185+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:10:42.752185+0000","last_peered":"2024-09-21T23:10:42.752185+0000","last_clean":"2024-09-21T23:10:42.752185+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:10:42.752185+0000","last_undegraded":"2024-09-21T23:10:42.752185+0000","last_fullsized":"2024-09-21T23:10:42.752185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:42.750890+0000","last_change":"2024-09-21T23:10:39.717465+0000","last_active":"2024-09-21T23:10:42.750890+0000","last_peered":"2024-09-21T23:10:42.750890+0000","last_clean":"2024-09-21T23:10:42.750890+0000","last_became_active":"2024-09-21T23:10:39.716861+0000","last_became_peered":"2024-09-21T23:10:39.716861+0000","last_unstale":"2024-09-21T23:10:42.750890+0000","last_undegraded":"2024-09-21T23:10:42.750890+0000","last_fullsized":"2024-09-21T23:10:42.750890+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:45.325732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"56'136","reported_seq":588,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.541723+0000","last_change":"2024-09-21T23:09:57.367811+0000","last_active":"2024-09-21T23:10:44.541723+0000","last_peered":"2024-09-21T23:10:44.541723+0000","last_clean":"2024-09-21T23:10:44.541723+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:10:44.541723+0000","last_undegraded":"2024-09-21T23:10:44.541723+0000","last_fullsized":"2024-09-21T23:10:44.541723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:18:23.285635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094978,"stat_sum":{"num_bytes":3532001,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":18,"num_read":1067,"num_read_kb":1346,"num_write":128,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"56'316","reported_seq":2217,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.802211+0000","last_change":"2024-09-21T23:10:09.464228+0000","last_active":"2024-09-21T23:10:44.802211+0000","last_peered":"2024-09-21T23:10:44.802211+0000","last_clean":"2024-09-21T23:10:04.423615+0000","last_became_active":"2024-09-21T23:10:09.464228+0000","last_became_peered":"2024-09-21T23:10:09.464228+0000","last_unstale":"2024-09-21T23:10:44.802211+0000","last_undegraded":"2024-09-21T23:10:09.462207+0000","last_fullsized":"2024-09-21T23:10:09.462006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:11.810104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084957200000000004,"stat_sum":{"num_bytes":21452561,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":4064,"num_read_kb":9734,"num_write":293,"num_write_kb":3226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":36}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"56'229","reported_seq":1961,"reported_epoch":56,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:10:44.837723+0000","last_change":"2024-09-21T23:10:09.471956+0000","last_active":"2024-09-21T23:10:44.837723+0000","last_peered":"2024-09-21T23:10:44.837723+0000","last_clean":"2024-09-21T23:10:07.736929+0000","last_became_active":"2024-09-21T23:10:09.471956+0000","last_became_peered":"2024-09-21T23:10:09.471956+0000","last_unstale":"2024-09-21T23:10:44.837723+0000","last_undegraded":"2024-09-21T23:10:09.470421+0000","last_fullsized":"2024-09-21T23:10:09.470194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:28:03.636754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073170699999999995,"stat_sum":{"num_bytes":26306074,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":2278,"num_read_kb":23705,"num_write":210,"num_write_kb":6889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":52}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"56'133","reported_seq":818,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.880447+0000","last_change":"2024-09-21T23:09:57.368899+0000","last_active":"2024-09-21T23:10:44.880447+0000","last_peered":"2024-09-21T23:10:44.880447+0000","last_clean":"2024-09-21T23:10:44.880447+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:10:44.880447+0000","last_undegraded":"2024-09-21T23:10:44.880447+0000","last_fullsized":"2024-09-21T23:10:44.880447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:36:23.236668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014237099999999999,"stat_sum":{"num_bytes":21823662,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":695,"num_read_kb":16959,"num_write":123,"num_write_kb":7322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"56'265","reported_seq":1002,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.883355+0000","last_change":"2024-09-21T23:09:57.379066+0000","last_active":"2024-09-21T23:10:44.883355+0000","last_peered":"2024-09-21T23:10:44.883355+0000","last_clean":"2024-09-21T23:10:44.883355+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:10:44.883355+0000","last_undegraded":"2024-09-21T23:10:44.883355+0000","last_fullsized":"2024-09-21T23:10:44.883355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:39:06.669829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050204199999999996,"stat_sum":{"num_bytes":18388623,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":27,"num_read":1450,"num_read_kb":12100,"num_write":279,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"56'187","reported_seq":1977,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.898309+0000","last_change":"2024-09-21T23:10:39.717493+0000","last_active":"2024-09-21T23:10:44.898309+0000","last_peered":"2024-09-21T23:10:44.898309+0000","last_clean":"2024-09-21T23:10:44.898309+0000","last_became_active":"2024-09-21T23:10:39.716583+0000","last_became_peered":"2024-09-21T23:10:39.716583+0000","last_unstale":"2024-09-21T23:10:44.898309+0000","last_undegraded":"2024-09-21T23:10:44.898309+0000","last_fullsized":"2024-09-21T23:10:44.898309+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:10:20.453316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010715919999999999,"stat_sum":{"num_bytes":52078680,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":28,"num_read":1572,"num_read_kb":19569,"num_write":166,"num_write_kb":10106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":9340862,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"56'149","reported_seq":1416,"reported_epoch":56,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:10:44.881736+0000","last_change":"2024-09-21T23:10:39.720341+0000","last_active":"2024-09-21T23:10:44.881736+0000","last_peered":"2024-09-21T23:10:44.881736+0000","last_clean":"2024-09-21T23:10:44.881736+0000","last_became_active":"2024-09-21T23:10:39.719313+0000","last_became_peered":"2024-09-21T23:10:39.719313+0000","last_unstale":"2024-09-21T23:10:44.881736+0000","last_undegraded":"2024-09-21T23:10:44.881736+0000","last_fullsized":"2024-09-21T23:10:44.881736+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:44.321818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010461890000000001,"stat_sum":{"num_bytes":46359801,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":1576,"num_read_kb":14989,"num_write":200,"num_write_kb":6403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":3602452,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[{"shards":"1,7","objects":39}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"56'343","reported_seq":1198,"reported_epoch":56,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:10:44.581522+0000","last_change":"2024-09-21T23:10:39.720649+0000","last_active":"2024-09-21T23:10:44.581522+0000","last_peered":"2024-09-21T23:10:44.581522+0000","last_clean":"2024-09-21T23:10:44.581522+0000","last_became_active":"2024-09-21T23:10:39.719489+0000","last_became_peered":"2024-09-21T23:10:39.719489+0000","last_unstale":"2024-09-21T23:10:44.581522+0000","last_undegraded":"2024-09-21T23:10:44.581522+0000","last_fullsized":"2024-09-21T23:10:44.581522+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:30:46.186345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010405119999999999,"stat_sum":{"num_bytes":17537970,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":31,"num_read":1550,"num_read_kb":13642,"num_write":451,"num_write_kb":6437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4752958,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[{"shards":"0,7","objects":54}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"56'260","reported_seq":1596,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.930676+0000","last_change":"2024-09-21T23:09:57.494767+0000","last_active":"2024-09-21T23:10:44.930676+0000","last_peered":"2024-09-21T23:10:44.930676+0000","last_clean":"2024-09-21T23:10:44.930676+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:10:44.930676+0000","last_undegraded":"2024-09-21T23:10:44.930676+0000","last_fullsized":"2024-09-21T23:10:44.930676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:50:44.013890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071189699999999996,"stat_sum":{"num_bytes":49990779,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":2403,"num_read_kb":20823,"num_write":306,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"56'202","reported_seq":1662,"reported_epoch":56,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:10:44.839276+0000","last_change":"2024-09-21T23:10:39.723037+0000","last_active":"2024-09-21T23:10:44.839276+0000","last_peered":"2024-09-21T23:10:44.839276+0000","last_clean":"2024-09-21T23:10:44.839276+0000","last_became_active":"2024-09-21T23:10:39.718469+0000","last_became_peered":"2024-09-21T23:10:39.718469+0000","last_unstale":"2024-09-21T23:10:44.839276+0000","last_undegraded":"2024-09-21T23:10:44.839276+0000","last_fullsized":"2024-09-21T23:10:44.839276+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:00:42.419137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010743459999999999,"stat_sum":{"num_bytes":59888655,"num_objects":42,"num_object_clones":5,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":1293,"num_read_kb":21513,"num_write":212,"num_write_kb":9526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":6607325,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":42}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"56'196","reported_seq":2011,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-21T23:10:44.935429+0000","last_change":"2024-09-21T23:10:39.717974+0000","last_active":"2024-09-21T23:10:44.935429+0000","last_peered":"2024-09-21T23:10:44.935429+0000","last_clean":"2024-09-21T23:10:44.935429+0000","last_became_active":"2024-09-21T23:10:39.716885+0000","last_became_peered":"2024-09-21T23:10:39.716885+0000","last_unstale":"2024-09-21T23:10:44.935429+0000","last_undegraded":"2024-09-21T23:10:44.935429+0000","last_fullsized":"2024-09-21T23:10:44.935429+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:09:17.636447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021982479999999999,"stat_sum":{"num_bytes":21979199,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":2395,"num_read_kb":23301,"num_write":168,"num_write_kb":13519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":12756271,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"56'153","reported_seq":681,"reported_epoch":56,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:10:44.805749+0000","last_change":"2024-09-21T23:10:39.721082+0000","last_active":"2024-09-21T23:10:44.805749+0000","last_peered":"2024-09-21T23:10:44.805749+0000","last_clean":"2024-09-21T23:10:44.805749+0000","last_became_active":"2024-09-21T23:10:39.718554+0000","last_became_peered":"2024-09-21T23:10:39.718554+0000","last_unstale":"2024-09-21T23:10:44.805749+0000","last_undegraded":"2024-09-21T23:10:44.805749+0000","last_fullsized":"2024-09-21T23:10:44.805749+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:11:54.405318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":15427923,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":478,"num_read_kb":8752,"num_write":135,"num_write_kb":4428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3666703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[{"shards":"2,7","objects":39}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":993578442,"num_objects":1329,"num_object_clones":203,"num_object_copies":2658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":330,"num_objects_misplaced":252,"num_objects_unfound":0,"num_objects_dirty":1329,"num_whiteouts":720,"num_read":62344,"num_read_kb":446120,"num_write":9288,"num_write_kb":232576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":59377131,"num_keys_recovered":47,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339705856,"data_stored":362136884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7823,"ondisk_log_size":7823,"up":50,"acting":56,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":15,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":55834574866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73128,"kb_used_data":41936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298712,"statfs":{"total":96636764160,"available":96561881088,"internally_reserved":0,"allocated":42942464,"data_stored":44039510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84248,"kb_used_data":48904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94287592,"statfs":{"total":96636764160,"available":96550494208,"internally_reserved":0,"allocated":50077696,"data_stored":57295589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1024000,"data_stored":796106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107756,"kb_used_data":68040,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94264084,"statfs":{"total":96636764160,"available":96526422016,"internally_reserved":0,"allocated":69672960,"data_stored":73931443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50164,"kb_used_data":18980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321676,"statfs":{"total":96636764160,"available":96585396224,"internally_reserved":0,"allocated":19435520,"data_stored":20688118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75552,"kb_used_data":40208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94296288,"statfs":{"total":96636764160,"available":96559398912,"internally_reserved":0,"allocated":41172992,"data_stored":43246118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119788,"kb_used_data":84440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94252052,"statfs":{"total":96636764160,"available":96514101248,"internally_reserved":0,"allocated":86466560,"data_stored":91016368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":85225472,"data_stored":90136694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40529920,"data_stored":42964985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18788352,"data_stored":20398812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":70119424,"data_stored":74727657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7651328,"data_stored":7610028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49426432,"data_stored":57006264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":41701376,"data_stored":43159836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":26263552,"data_stored":26132608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:10:47.592 INFO:tasks.thrashosds.thrasher:Setting 3.18 to [3, 2, 7, 4] 2024-09-21T23:10:47.592 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.18', '3', '2', '7', '4'] 2024-09-21T23:10:47.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.18 3 2 7 4 2024-09-21T23:10:47.596 INFO:teuthology.orchestra.run.smithi150.stdout:2842 clone 26 order 18 su 32768 sc 13 2024-09-21T23:10:47.603 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:47.603+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:47.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:47.679 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:47.679 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:47.692 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:47.692+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:47.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:47.771 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:47.768+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:47.775 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:10:47.793 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:47.793 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:47.822 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:47.822 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:47.839 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:47.839 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:47.839 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:47.839 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:47.844 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:47.844 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:47.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:47.864 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:47.865 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:47.865 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:47.872 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:47.871+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:47.892 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:47.892 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:47.904 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone25 from 0x5c25482 (overlap 0x5c25482) to 0x49b76ce 2024-09-21T23:10:47.924 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:47.924 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:47.952 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:47.952 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:47.955 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:47.956 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:47.998 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:47.998 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:47.999 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:47.999 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:47.999 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:47.999 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:47.996+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:48.015 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:48.104 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:48.104+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:48.120 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-21T23:10:48.174 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:48.172+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:48.210 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:48.210 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:48.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:48.210 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:48.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6057897399999996, 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.211 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:10:48.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:10:48.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6057430730000002, 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.219 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6056690070000004, 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:10:48.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:10:48.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:10:48.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.223 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.605606281, 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6055543310000004, 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:10:48.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.605341245, 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6034572200000001, 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.236 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6034091699999999, 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.240 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6033608629999998, 2024-09-21T23:10:48.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:10:48.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:10:48.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.248 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6033136480000003, 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.603249419, 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:10:48.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6031975090000001, 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.257 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:10:48.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40186 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 442368~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5986499070000004, 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093234290999999997, 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40186 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610104+0000", 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610109+0000", 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2140000000000004e-06 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610125+0000", 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5276000000000001e-05 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610139+0000", 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4477e-05 2024-09-21T23:10:48.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703148+0000", 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093008570999999998 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703236+0000", 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7868000000000006e-05 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703337+0000", 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101742 2024-09-21T23:10:48.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5985952149999996, 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:10:48.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:10:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5985404000000001, 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:10:48.270 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:10:48.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:10:48.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5984926760000002, 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:10:48.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:10:48.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:10:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.598425787, 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5948805090000002, 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:10:48.287 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.594833768, 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:10:48.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.291 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5947872460000001, 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.296 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.296 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.296 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:48.297 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:48.297 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:48.295+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:48.398 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:48.395+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:48.498 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:48.495+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:48.500 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:48.500 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:48.517 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:48.556 INFO:teuthology.orchestra.run.smithi150.stdout:2844 trunc from 0x5c25482 to 0xdbbe0a1 2024-09-21T23:10:48.567 INFO:teuthology.orchestra.run.smithi150.stdout:2845 read 0x2db2c00 thru 0x2dc29bd (0xfdbe bytes) 2024-09-21T23:10:48.568 INFO:teuthology.orchestra.run.smithi150.stdout:2846 read 0x1b7f7ff thru 0x1b884ab (0x8cad bytes) 2024-09-21T23:10:48.569 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:48.569 INFO:teuthology.orchestra.run.smithi150.stdout:2848 read 0xb3b65b thru 0xb43bf2 (0x8598 bytes) 2024-09-21T23:10:48.575 INFO:teuthology.orchestra.run.smithi150.stdout:2849 write 0xdc373d2 thru 0xdc4175d (0xa38c bytes) 2024-09-21T23:10:48.589 INFO:teuthology.orchestra.run.smithi150.stdout:2850 writesame 0x4d63c1a thru 0x4d6dca0 data_size 0x1(0xa087 bytes) 2024-09-21T23:10:48.599 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:48.596+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:48.619 INFO:teuthology.orchestra.run.smithi150.stdout:2852 read 0x2391c99 thru 0x2395bf5 (0x3f5d bytes) 2024-09-21T23:10:48.620 INFO:teuthology.orchestra.run.smithi150.stdout:2853 write 0x93f6031 thru 0x94012ec (0xb2bc bytes) 2024-09-21T23:10:48.632 INFO:teuthology.orchestra.run.smithi150.stdout:2854 trunc from 0xdc4175e to 0xc15df15 2024-09-21T23:10:48.644 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:48.644 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:48.644 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:48.644 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:48.644 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:48.649 INFO:teuthology.orchestra.run.smithi150.stdout:2855 read 0x2ca7e4e thru 0x2cb2be4 (0xad97 bytes) 2024-09-21T23:10:48.650 INFO:teuthology.orchestra.run.smithi150.stdout:2856 write 0x52d6157 thru 0x52d6661 (0x50b bytes) 2024-09-21T23:10:48.656 INFO:teuthology.orchestra.run.smithi150.stdout:2857 punch from 0x1d272d6 to 0x1d3587d, (0xe5a7 bytes) 2024-09-21T23:10:48.657 INFO:teuthology.orchestra.run.smithi150.stdout:2858 punch from 0x8fa9de9 to 0x8facfb1, (0x31c8 bytes) 2024-09-21T23:10:48.657 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:48.657 INFO:teuthology.orchestra.run.smithi150.stdout:2861 punch from 0xc0c9d00 to 0xc0cc169, (0x2469 bytes) 2024-09-21T23:10:48.657 INFO:teuthology.orchestra.run.smithi150.stdout:2862 punch from 0xc0e6e57 to 0xc0f0af4, (0x9c9d bytes) 2024-09-21T23:10:48.657 INFO:teuthology.orchestra.run.smithi150.stdout:2865 punch from 0x33eedcd to 0x33f80f6, (0x9329 bytes) 2024-09-21T23:10:48.657 INFO:teuthology.orchestra.run.smithi150.stdout:2868 write 0x7346561 thru 0x734fe4b (0x98eb bytes) 2024-09-21T23:10:48.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:48.665 INFO:teuthology.orchestra.run.smithi150.stdout:2869 write 0x436c44 thru 0x43ba66 (0x4e23 bytes) 2024-09-21T23:10:48.677 INFO:teuthology.orchestra.run.smithi150.stdout:2872 punch from 0x98006b8 to 0x980da64, (0xd3ac bytes) 2024-09-21T23:10:48.678 INFO:teuthology.orchestra.run.smithi150.stdout:2873 read 0x33a553a thru 0x33acb47 (0x760e bytes) 2024-09-21T23:10:48.678 INFO:teuthology.orchestra.run.smithi150.stdout:2875 read 0x1a93954 thru 0x1a948cb (0xf78 bytes) 2024-09-21T23:10:48.678 INFO:teuthology.orchestra.run.smithi150.stdout:2877 read 0x941c2da thru 0x9420c86 (0x49ad bytes) 2024-09-21T23:10:48.678 INFO:teuthology.orchestra.run.smithi150.stdout:2879 read 0x83a37cf thru 0x83a425a (0xa8c bytes) 2024-09-21T23:10:48.678 INFO:teuthology.orchestra.run.smithi150.stdout:2881 write 0xb84c74f thru 0xb858ad0 (0xc382 bytes) 2024-09-21T23:10:48.687 INFO:teuthology.orchestra.run.smithi150.stdout:2882 punch from 0x368fa3a to 0x36984b7, (0x8a7d bytes) 2024-09-21T23:10:48.687 INFO:teuthology.orchestra.run.smithi150.stdout:2884 write 0x4e5391e thru 0x4e56af8 (0x31db bytes) 2024-09-21T23:10:48.693 INFO:teuthology.orchestra.run.smithi150.stdout:2885 write 0x1bf6ae1 thru 0x1c01df2 (0xb312 bytes) 2024-09-21T23:10:48.700 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:48.699+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:48.705 INFO:teuthology.orchestra.run.smithi150.stdout:2886 read 0x899f9ef thru 0x89a13d5 (0x19e7 bytes) 2024-09-21T23:10:48.707 INFO:teuthology.orchestra.run.smithi150.stdout:2887 trunc from 0xc15df15 to 0xd381a30 2024-09-21T23:10:48.714 INFO:teuthology.orchestra.run.smithi150.stdout:2890 read 0x234d103 thru 0x2353058 (0x5f56 bytes) 2024-09-21T23:10:48.714 INFO:teuthology.orchestra.run.smithi150.stdout:2892 read 0xcacf2dd thru 0xcadb9db (0xc6ff bytes) 2024-09-21T23:10:48.714 INFO:teuthology.orchestra.run.smithi150.stdout:2894 writesame 0x98226cd thru 0x9826167 data_size 0x3(0x3a9b bytes) 2024-09-21T23:10:48.720 INFO:teuthology.orchestra.run.smithi150.stdout:2895 read 0x8185e2e thru 0x8187fed (0x21c0 bytes) 2024-09-21T23:10:48.728 INFO:teuthology.orchestra.run.smithi150.stdout:2896 trunc from 0xd381a30 to 0x6aa056b 2024-09-21T23:10:48.745 INFO:teuthology.orchestra.run.smithi150.stdout:2897 write 0x7237878 thru 0x723cd0d (0x5496 bytes) 2024-09-21T23:10:48.767 INFO:teuthology.orchestra.run.smithi150.stdout:2898 write 0xe939877 thru 0xe93f5fb (0x5d85 bytes) 2024-09-21T23:10:48.777 INFO:teuthology.orchestra.run.smithi150.stdout:2899 trunc from 0xe93f5fc to 0xa304033 2024-09-21T23:10:48.784 INFO:teuthology.orchestra.run.smithi150.stderr:set 3.18 pg_upmap_items mapping to [3->2,7->4] 2024-09-21T23:10:48.784 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:10:48.801 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:48.799+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:48.803 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:48.803 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:48.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51023 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56692019999999999, 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037458331999999997, 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51023 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:10:48.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235893+0000", 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5100000000000001e-07 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235895+0000", 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3240000000000001e-06 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235901+0000", 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0610000000000001e-06 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235906+0000", 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1830000000000002e-06 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272599+0000", 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036692726000000002 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272669+0000", 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9417000000000003e-05 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273351+0000", 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00068227000000000001 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51024 3.4 3:2264aa58:::rbd_header.1149796cbcee:head [watch reconnect cookie 139807362363424 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:10:48.808 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56690033900000003, 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039816437000000003, 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51024 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235913+0000", 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235915+0000", 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.897e-06 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235921+0000", 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3759999999999996e-06 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235926+0000", 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7759999999999997e-06 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273380+0000", 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037453792999999999 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273460+0000", 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0241999999999995e-05 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275729+0000", 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022688740000000002 2024-09-21T23:10:48.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51026 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56688086900000001, 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038933657000000003, 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51026 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.41e-07 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235935+0000", 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6369999999999999e-06 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235941+0000", 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.528e-06 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235947+0000", 2024-09-21T23:10:48.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2959999999999998e-06 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274454+0000", 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038507844999999999 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274533+0000", 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.839e-05 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274865+0000", 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033262 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51028 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56686008200000004, 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03952514, 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51028 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235955+0000", 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0640000000000001e-06 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235961+0000", 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1099999999999999e-06 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235966+0000", 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4120000000000003e-06 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274896+0000", 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038930017999999997 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274973+0000", 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7616999999999999e-05 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275478+0000", 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000504544 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51030 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.566841292, 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040194278, 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51030 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235972+0000", 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235974+0000", 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3159999999999999e-06 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235980+0000", 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235985+0000", 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0359999999999997e-06 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275761+0000", 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776211999999998 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275838+0000", 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7231999999999995e-05 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276166+0000", 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032743300000000002 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51056 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56673994299999997, 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040641749999999997, 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51056 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.829 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:10:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236080+0000", 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0180000000000004e-06 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236087+0000", 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8759999999999999e-06 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236092+0000", 2024-09-21T23:10:48.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276196+0000", 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040103449999999999 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276272+0000", 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6062000000000005e-05 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276715+0000", 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000442719 2024-09-21T23:10:48.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51057 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56671337799999999, 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041665445000000002, 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51057 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236100+0000", 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236102+0000", 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0990000000000001e-06 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236110+0000", 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1610000000000003e-06 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236115+0000", 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9880000000000004e-06 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276744+0000", 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040628876000000001 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276810+0000", 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6359999999999995e-05 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277765+0000", 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00095440200000000003 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51058 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56669082500000001, 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041431192999999998, 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51058 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236125+0000", 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:10:48.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236131+0000", 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4489999999999996e-06 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236136+0000", 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277067+0000", 2024-09-21T23:10:48.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040930381000000002 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277212+0000", 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014488599999999999 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277553+0000", 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000341424 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51059 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.56667010399999995, 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042250247999999997, 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51059 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:10:48.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236145+0000", 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236151+0000", 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.268e-06 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236157+0000", 2024-09-21T23:10:48.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4480000000000002e-06 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277579+0000", 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041421948 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277693+0000", 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113786 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.278393+0000", 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00070025299999999996 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51323 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252833+0000", 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.549979569, 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037218182000000002, 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51323 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252833+0000", 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252833+0000", 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252834+0000", 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252836+0000", 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2050000000000001e-06 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252843+0000", 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7329999999999997e-06 2024-09-21T23:10:48.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252848+0000", 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3319999999999999e-06 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.289891+0000", 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037043485000000001 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.289963+0000", 2024-09-21T23:10:48.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2139999999999997e-05 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290051+0000", 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7837999999999995e-05 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51324 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54995822599999999, 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037384598999999998, 2024-09-21T23:10:48.850 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51324 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252855+0000", 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252857+0000", 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2709999999999999e-06 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252864+0000", 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6889999999999998e-06 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252869+0000", 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1429999999999997e-06 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290081+0000", 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037211602000000003 2024-09-21T23:10:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290154+0000", 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3200000000000004e-05 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290239+0000", 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5191000000000005e-05 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51325 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54993679600000001, 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037730817, 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51325 2024-09-21T23:10:48.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252879+0000", 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2239999999999998e-06 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252885+0000", 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3010000000000003e-06 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252890+0000", 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3569999999999997e-06 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290267+0000", 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037376615000000002 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290338+0000", 2024-09-21T23:10:48.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1265000000000003e-05 2024-09-21T23:10:48.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290607+0000", 2024-09-21T23:10:48.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026868200000000001 2024-09-21T23:10:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51338 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54983288500000005, 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038044604000000003, 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51338 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8099999999999998e-07 2024-09-21T23:10:48.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252982+0000", 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3020000000000002e-06 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252989+0000", 2024-09-21T23:10:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3149999999999997e-06 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252994+0000", 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9359999999999998e-06 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290639+0000", 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037645064999999998 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290713+0000", 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4115000000000004e-05 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291024+0000", 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031148999999999999 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51339 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54981144000000004, 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038794718999999998, 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51339 2024-09-21T23:10:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253002+0000", 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253005+0000", 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253011+0000", 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4949999999999996e-06 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253016+0000", 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291053+0000", 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038037165999999997 2024-09-21T23:10:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291126+0000", 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2575000000000002e-05 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291796+0000", 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067037100000000001 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51340 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54977697299999995, 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039295921999999997, 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51340 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9699999999999996e-07 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253039+0000", 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5220000000000002e-06 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253046+0000", 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253051+0000", 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1869999999999996e-06 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291828+0000", 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038776772000000001 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291914+0000", 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5847999999999995e-05 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292332+0000", 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000418136 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51341 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54975496400000001, 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039551238000000002, 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51341 2024-09-21T23:10:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:10:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253064+0000", 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253072+0000", 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0339999999999997e-06 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253077+0000", 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.921e-06 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292363+0000", 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039285863999999997 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292437+0000", 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4247999999999998e-05 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292609+0000", 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017169900000000001 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51475 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54685396200000003, 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037285393999999999, 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51475 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:10:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255960+0000", 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255965+0000", 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8760000000000004e-06 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255977+0000", 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2629000000000001e-05 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255985+0000", 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1179999999999992e-06 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292638+0000", 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036653036 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292713+0000", 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4262000000000002e-05 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293244+0000", 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00053145700000000004 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51477 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54681290299999996, 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037740821000000001, 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51477 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:10:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9599999999999999e-07 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256003+0000", 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.627e-06 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256011+0000", 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256016+0000", 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.784e-06 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293276+0000", 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037260003 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293363+0000", 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6558999999999996e-05 2024-09-21T23:10:48.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293741+0000", 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037767200000000001 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.889 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51479 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54678826700000005, 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038089583000000003, 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51479 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256025+0000", 2024-09-21T23:10:48.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.13e-07 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256027+0000", 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1399999999999998e-06 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256034+0000", 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256038+0000", 2024-09-21T23:10:48.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293776+0000", 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037737241999999997 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293852+0000", 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6287999999999999e-05 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294114+0000", 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026198699999999998 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51481 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.54676723699999996, 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038735549000000001, 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:48.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51481 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:10:48.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256046+0000", 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256048+0000", 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2359999999999999e-06 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256055+0000", 2024-09-21T23:10:48.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6669999999999999e-06 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256061+0000", 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6200000000000004e-06 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294146+0000", 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038085073999999997 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294234+0000", 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8839999999999996e-05 2024-09-21T23:10:48.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294781+0000", 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00054657999999999996 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:48.898 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:48.901 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:48.902 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:48.899+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:49.003 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:49.003+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:49.067 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.27890874700000001, 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.27895365900000002, 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for peered", 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:10:49.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:10:49.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:10:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 1 2024-09-21T23:10:49.072 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:49.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:49.128 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:49.128+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:49.199 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:49.199 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:49.199 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:49.199 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:49.199 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:49.205 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:49.203+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:49.214 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:49.306 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:49.304+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:49.378 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:49.378 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39773 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8364032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491489+0000", 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8861743290000001, 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019142147000000002, 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39773 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491489+0000", 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491489+0000", 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491490+0000", 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.144e-06 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491497+0000", 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3860000000000004e-06 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491516+0000", 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9276e-05 2024-09-21T23:10:49.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491530+0000", 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3653e-05 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510153+0000", 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018623520000000001 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510375+0000", 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022212800000000001 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510631+0000", 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025604000000000002 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8861169640000002, 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:10:49.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.886063526, 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:10:49.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8860125510000003, 2024-09-21T23:10:49.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:10:49.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8859616199999998, 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:10:49.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:10:49.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8859063090000001, 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:10:49.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:10:49.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8858373940000002, 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.407 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8812000409999996, 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:10:49.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:10:49.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:10:49.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.8811318999999997, 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:10:49.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3949783890000003, 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:10:49.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:10:49.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:10:49.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41320 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9289728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983670+0000", 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3939931249999997, 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019152985000000001, 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41320 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983670+0000", 2024-09-21T23:10:49.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983670+0000", 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983672+0000", 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983676+0000", 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3159999999999998e-06 2024-09-21T23:10:49.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983688+0000", 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2493e-05 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983698+0000", 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0152e-05 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002623+0000", 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018924252999999999 2024-09-21T23:10:49.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002701+0000", 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8372000000000001e-05 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002823+0000", 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012227900000000001 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3939250139999997, 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:49.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3938786470000002, 2024-09-21T23:10:49.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:10:49.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.431 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3937504680000004, 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:10:49.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:10:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:10:49.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.393664673, 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:10:49.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:10:49.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3931208579999996, 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:10:49.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:10:49.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3930230779999997, 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.444 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3929482039999996, 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3928874259999997, 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:10:49.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3928511080000003, 2024-09-21T23:10:49.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.460 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:49.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:49.462 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:49.460+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:49.562 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:49.563+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:49.632 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:49.632 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:49.646 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:49.692 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:49.692+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:49.763 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:49.763 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:49.763 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:49.763 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:49.763 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:49.763 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:49.760+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:49.777 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:49.864 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:49.863+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:49.947 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.870342409000003, 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:49.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:49.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.704575558000002, 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:49.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:49.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:38234 3.1e 3:7a5571f8:::rbd_data.1149ed0ffb75.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.9766304039999998, 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043458748999999998, 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.958 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 38234 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968400+0000", 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6360000000000001e-06 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968406+0000", 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.81e-06 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968419+0000", 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3046e-05 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968426+0000", 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5440000000000004e-06 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968519+0000", 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3348000000000001e-05 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968668+0000", 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148264 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.977244+0000", 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085759600000000005 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011662+0000", 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034417917999999999 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011690+0000", 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8368999999999999e-05 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011780+0000", 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0450000000000003e-05 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011857+0000", 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6403999999999996e-05 2024-09-21T23:10:49.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52271 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.586669219, 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039664149000000003, 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52271 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358360+0000", 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358362+0000", 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9860000000000001e-06 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358368+0000", 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:10:49.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358372+0000", 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7859999999999996e-06 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398016+0000", 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039643954000000002 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398023+0000", 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9249999999999998e-06 2024-09-21T23:10:49.967 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52394 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.586403043, 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039669796, 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.968 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52394 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358626+0000", 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6800000000000001e-07 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358628+0000", 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:10:49.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358634+0000", 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358639+0000", 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.772e-06 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398288+0000", 2024-09-21T23:10:49.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039648640999999998 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398295+0000", 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0720000000000002e-06 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52395 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5863819110000001, 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039678049999999999, 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.972 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52395 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358647+0000", 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358649+0000", 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569999999999998e-06 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358655+0000", 2024-09-21T23:10:49.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.127e-06 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358660+0000", 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6820000000000002e-06 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398317+0000", 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039657215000000003 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398325+0000", 2024-09-21T23:10:49.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4789999999999999e-06 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52396 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5863619739999999, 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039688093000000001, 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52396 2024-09-21T23:10:49.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358667+0000", 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:10:49.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358669+0000", 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1509999999999998e-06 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358676+0000", 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-06 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358681+0000", 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1420000000000001e-06 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398348+0000", 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039666202999999997 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398355+0000", 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.861e-06 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.979 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52397 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5863359909999999, 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039702474000000001, 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52397 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358693+0000", 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358695+0000", 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358701+0000", 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358706+0000", 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8550000000000001e-06 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398385+0000", 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039679157999999999 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398395+0000", 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9299999999999998e-06 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52398 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.983 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.586314676, 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039712074999999999, 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52398 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358716+0000", 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0940000000000002e-06 2024-09-21T23:10:49.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358722+0000", 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-06 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358727+0000", 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0760000000000002e-06 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398418+0000", 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039691048 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398426+0000", 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52399 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5862947649999999, 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039729353000000002, 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.987 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.988 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52399 2024-09-21T23:10:49.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358736+0000", 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358743+0000", 2024-09-21T23:10:49.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4119999999999998e-06 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358747+0000", 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8749999999999999e-06 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398453+0000", 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039705424000000003 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398463+0000", 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0182e-05 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52432 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.586274787, 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039740801999999999, 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52432 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:49.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358756+0000", 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358762+0000", 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:10:49.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358767+0000", 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6129999999999997e-06 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398487+0000", 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039720278999999997 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398494+0000", 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5619999999999998e-06 2024-09-21T23:10:49.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52433 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5862541860000001, 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039750774000000003, 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52433 2024-09-21T23:10:49.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358775+0000", 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:10:49.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358777+0000", 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358783+0000", 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358788+0000", 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.135e-06 2024-09-21T23:10:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398518+0000", 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039730389999999997 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398525+0000", 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52434 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.586234417, 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039771847999999999, 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52434 2024-09-21T23:10:50.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358796+0000", 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8899999999999999e-06 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358802+0000", 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7139999999999998e-06 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358807+0000", 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8029999999999996e-06 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398556+0000", 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039749022000000002 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398566+0000", 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0067e-05 2024-09-21T23:10:50.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52435 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5862147900000001, 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039820863999999997, 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52435 2024-09-21T23:10:50.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:50.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8599999999999999e-07 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358816+0000", 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2180000000000002e-06 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358822+0000", 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9939999999999997e-06 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358827+0000", 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7670000000000003e-06 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398624+0000", 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797341 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398634+0000", 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0158e-05 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52499 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.008 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.585926138, 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039762180000000001, 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52499 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359103+0000", 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359105+0000", 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359111+0000", 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1120000000000001e-06 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359116+0000", 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398857+0000", 2024-09-21T23:10:50.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039741132999999998 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398864+0000", 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5870000000000004e-06 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52500 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5859062800000001, 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776995000000002, 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.012 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52500 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:50.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359123+0000", 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359124+0000", 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9750000000000001e-06 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359131+0000", 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-21T23:10:50.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359135+0000", 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7520000000000004e-06 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398892+0000", 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039756315 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398899+0000", 2024-09-21T23:10:50.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.43e-06 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52526 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.585887002, 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039791576000000002, 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.016 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52526 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359142+0000", 2024-09-21T23:10:50.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359144+0000", 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.838e-06 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359150+0000", 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359154+0000", 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398925+0000", 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039770497000000002 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398933+0000", 2024-09-21T23:10:50.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1319999999999994e-06 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52527 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.58586776, 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03980852, 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52527 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359163+0000", 2024-09-21T23:10:50.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.995e-06 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359169+0000", 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9710000000000001e-06 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359174+0000", 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0629999999999996e-06 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398961+0000", 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039786948000000003 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398969+0000", 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1580000000000005e-06 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52528 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.585848159, 2024-09-21T23:10:50.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039822428999999999, 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52528 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:50.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359181+0000", 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359187+0000", 2024-09-21T23:10:50.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359192+0000", 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7509999999999997e-06 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359198+0000", 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9830000000000001e-06 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398995+0000", 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797036000000001 2024-09-21T23:10:50.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399003+0000", 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52529 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5858230180000001, 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039831105999999998, 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:50.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52529 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359206+0000", 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359208+0000", 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359214+0000", 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1e-06 2024-09-21T23:10:50.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359219+0000", 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7879999999999997e-06 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399029+0000", 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039810509000000001 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399036+0000", 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:10:50.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:50.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:50.032 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:50.032 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:50.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:50.034 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:50.032+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:50.134 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:50.132+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:50.171 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:50.171 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:50.171 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:50.171 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:50.184 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:50.224 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:50.224 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:50.235 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:50.235+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:50.240 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:50.241 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:50.257 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:50.257 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:50.273 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:50.274 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:50.274 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:50.292 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:50.313 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:50.313 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:50.328 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:50.371 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:50.371+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:50.437 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:50.436+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:50.489 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:50.489 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:50.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:50.515 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:50.516 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:50.538 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:50.538 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:50.543 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:50.543 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:50.547 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:50.547+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:50.624 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:50.624 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:50.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:50.639 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:50.636+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:50.740 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:50.740+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:50.800 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:50.800 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:50.813 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:50.859 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:50.859+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:50.900 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:10:50.931 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:50.931 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:50.942 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:50.939+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:50.945 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:51.043 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:51.043+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:51.112 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:51.112 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:51.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:51.171 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:51.171+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:51.244 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:51.243+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:51.290 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:51.290 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:51.303 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:51.334 INFO:teuthology.orchestra.run.smithi150.stdout:2900 punch from 0xf09dfd to 0xf16819, (0xca1c bytes) 2024-09-21T23:10:51.347 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:51.347+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:51.361 INFO:teuthology.orchestra.run.smithi150.stdout:2902 trunc from 0xa304033 to 0xc1b1a4 2024-09-21T23:10:51.446 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:51.444+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:51.458 INFO:teuthology.orchestra.run.smithi150.stdout:2903 punch from 0x253e09 to 0x26161b, (0xd812 bytes) 2024-09-21T23:10:51.458 INFO:teuthology.orchestra.run.smithi150.stdout:2904 read 0xc203f thru 0xcaee1 (0x8ea3 bytes) 2024-09-21T23:10:51.462 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.468 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:51.468 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:51.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:51.490 INFO:teuthology.orchestra.run.smithi150.stdout:2906 write 0xb8c1a42 thru 0xb8c572c (0x3ceb bytes) 2024-09-21T23:10:51.498 INFO:teuthology.orchestra.run.smithi150.stdout:2909 write 0x98f7f04 thru 0x9902989 (0xaa86 bytes) 2024-09-21T23:10:51.507 INFO:teuthology.orchestra.run.smithi150.stdout:2910 read 0x42defb0 thru 0x42e3e8e (0x4edf bytes) 2024-09-21T23:10:51.507 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.508 INFO:teuthology.orchestra.run.smithi150.stdout:2912 punch from 0x8659426 to 0x8668c0b, (0xf7e5 bytes) 2024-09-21T23:10:51.508 INFO:teuthology.orchestra.run.smithi150.stdout:2914 read 0x2819594 thru 0x281aee5 (0x1952 bytes) 2024-09-21T23:10:51.508 INFO:teuthology.orchestra.run.smithi150.stdout:2916 read 0xa9e8ea8 thru 0xa9e90bb (0x214 bytes) 2024-09-21T23:10:51.508 INFO:teuthology.orchestra.run.smithi150.stdout:2917 read 0x5541a5f thru 0x554268b (0xc2d bytes) 2024-09-21T23:10:51.508 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.511 INFO:teuthology.orchestra.run.smithi150.stdout:2920 trunc from 0xb8c572d to 0xdca0796 2024-09-21T23:10:51.518 INFO:teuthology.orchestra.run.smithi150.stdout:2921 writesame 0xa79893a thru 0xa7a182f data_size 0x1d(0x8ef6 bytes) 2024-09-21T23:10:51.523 INFO:teuthology.orchestra.run.smithi150.stdout:2924 read 0xaed117f thru 0xaedcfb7 (0xbe39 bytes) 2024-09-21T23:10:51.523 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.523 INFO:teuthology.orchestra.run.smithi150.stdout:2928 read 0x71bda99 thru 0x71c09e3 (0x2f4b bytes) 2024-09-21T23:10:51.523 INFO:teuthology.orchestra.run.smithi150.stdout:2929 read 0x16caab3 thru 0x16d03cb (0x5919 bytes) 2024-09-21T23:10:51.523 INFO:teuthology.orchestra.run.smithi150.stdout:2931 read 0x27cd1ac thru 0x27d2946 (0x579b bytes) 2024-09-21T23:10:51.523 INFO:teuthology.orchestra.run.smithi150.stdout:2932 read 0xab9aef0 thru 0xaba7743 (0xc854 bytes) 2024-09-21T23:10:51.533 INFO:teuthology.orchestra.run.smithi150.stdout:2933 trunc from 0xdca0796 to 0x6ac0458 2024-09-21T23:10:51.546 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:51.544+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:51.548 INFO:teuthology.orchestra.run.smithi150.stdout:2934 write 0x3752627 thru 0x3755c92 (0x366c bytes) 2024-09-21T23:10:51.552 INFO:teuthology.orchestra.run.smithi150.stdout:2935 read 0x196a264 thru 0x1978681 (0xe41e bytes) 2024-09-21T23:10:51.552 INFO:teuthology.orchestra.run.smithi150.stdout:2936 punch from 0x4e7c3b to 0x4e8584, (0x949 bytes) 2024-09-21T23:10:51.552 INFO:teuthology.orchestra.run.smithi150.stdout:2937 read 0x3d3d0f4 thru 0x3d3f147 (0x2054 bytes) 2024-09-21T23:10:51.570 INFO:teuthology.orchestra.run.smithi150.stdout:2938 write 0xdd4ef8a thru 0xdd53959 (0x49d0 bytes) 2024-09-21T23:10:51.576 INFO:teuthology.orchestra.run.smithi150.stdout:2939 read 0xc14754f thru 0xc1524d3 (0xaf85 bytes) 2024-09-21T23:10:51.576 INFO:teuthology.orchestra.run.smithi150.stdout:2940 punch from 0x175ae2 to 0x181fd7, (0xc4f5 bytes) 2024-09-21T23:10:51.576 INFO:teuthology.orchestra.run.smithi150.stdout:2941 read 0x328e1a3 thru 0x3291b8e (0x39ec bytes) 2024-09-21T23:10:51.589 INFO:teuthology.orchestra.run.smithi150.stdout:2943 trunc from 0xdd5395a to 0x4ee38c4 2024-09-21T23:10:51.603 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.615 INFO:teuthology.orchestra.run.smithi150.stdout:2947 trunc from 0x4ee38c4 to 0xc727e28 2024-09-21T23:10:51.623 INFO:teuthology.orchestra.run.smithi150.stdout:2948 read 0x7b2d6b2 thru 0x7b3c185 (0xead4 bytes) 2024-09-21T23:10:51.623 INFO:teuthology.orchestra.run.smithi150.stdout:2949 read 0x1e835c4 thru 0x1e85f7e (0x29bb bytes) 2024-09-21T23:10:51.623 INFO:teuthology.orchestra.run.smithi150.stdout:2950 read 0xcdb78c thru 0xce382b (0x80a0 bytes) 2024-09-21T23:10:51.624 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.624 INFO:teuthology.orchestra.run.smithi150.stdout:2953 write 0xc62de13 thru 0xc63d1fe (0xf3ec bytes) 2024-09-21T23:10:51.629 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:51.629 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:51.635 INFO:teuthology.orchestra.run.smithi150.stdout:2956 read 0xb0c78d9 thru 0xb0cbef7 (0x461f bytes) 2024-09-21T23:10:51.635 INFO:teuthology.orchestra.run.smithi150.stdout:2957 write 0xb26acc5 thru 0xb27ab6d (0xfea9 bytes) 2024-09-21T23:10:51.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:51.647 INFO:teuthology.orchestra.run.smithi150.stdout:2958 punch from 0xa8f5596 to 0xa9040ff, (0xeb69 bytes) 2024-09-21T23:10:51.647 INFO:teuthology.orchestra.run.smithi150.stdout:2959 write 0xb8ffb96 thru 0xb90f2bb (0xf726 bytes) 2024-09-21T23:10:51.647 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:51.647+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:51.657 INFO:teuthology.orchestra.run.smithi150.stdout:2961 write 0x8fee547 thru 0x8ffc731 (0xe1eb bytes) 2024-09-21T23:10:51.664 INFO:teuthology.orchestra.run.smithi150.stdout:2963 writesame 0x5045dd1 thru 0x50498fc data_size 0xe(0x3b2c bytes) 2024-09-21T23:10:51.669 INFO:teuthology.orchestra.run.smithi150.stdout:2964 read 0x25d4d2a thru 0x25e256a (0xd841 bytes) 2024-09-21T23:10:51.669 INFO:teuthology.orchestra.run.smithi150.stdout:2965 write 0x4ca699a thru 0x4cabde3 (0x544a bytes) 2024-09-21T23:10:51.674 INFO:teuthology.orchestra.run.smithi150.stdout:2966 read 0x7b0c3d8 thru 0x7b11224 (0x4e4d bytes) 2024-09-21T23:10:51.674 INFO:teuthology.orchestra.run.smithi150.stdout:2968 write 0x7e84c54 thru 0x7e91a0c (0xcdb9 bytes) 2024-09-21T23:10:51.683 INFO:teuthology.orchestra.run.smithi150.stdout:2969 punch from 0x6a99a17 to 0x6aa97b1, (0xfd9a bytes) 2024-09-21T23:10:51.683 INFO:teuthology.orchestra.run.smithi150.stdout:2970 read 0x542ca77 thru 0x5430921 (0x3eab bytes) 2024-09-21T23:10:51.683 INFO:teuthology.orchestra.run.smithi150.stdout:2971 read 0x595bf07 thru 0x5962ac5 (0x6bbf bytes) 2024-09-21T23:10:51.683 INFO:teuthology.orchestra.run.smithi150.stdout:2972 punch from 0x39f93ec to 0x3a04c70, (0xb884 bytes) 2024-09-21T23:10:51.683 INFO:teuthology.orchestra.run.smithi150.stdout:2973 write 0x5b7e7b4 thru 0x5b8b45e (0xccab bytes) 2024-09-21T23:10:51.692 INFO:teuthology.orchestra.run.smithi150.stdout:2975 punch from 0xb50a139 to 0xb51835f, (0xe226 bytes) 2024-09-21T23:10:51.692 INFO:teuthology.orchestra.run.smithi150.stdout:2977 writesame 0x8cfde41 thru 0x8d000c0 data_size 0x30(0x2280 bytes) 2024-09-21T23:10:51.699 INFO:teuthology.orchestra.run.smithi150.stdout:2978 read 0x7425a71 thru 0x7430413 (0xa9a3 bytes) 2024-09-21T23:10:51.699 INFO:teuthology.orchestra.run.smithi150.stdout:2980 read 0xad1db7e thru 0xad1eb15 (0xf98 bytes) 2024-09-21T23:10:51.699 INFO:teuthology.orchestra.run.smithi150.stdout:2981 read 0x66e4534 thru 0x66effb7 (0xba84 bytes) 2024-09-21T23:10:51.699 INFO:teuthology.orchestra.run.smithi150.stdout:2983 read 0x3f23242 thru 0x3f28450 (0x520f bytes) 2024-09-21T23:10:51.700 INFO:teuthology.orchestra.run.smithi150.stdout:2984 read 0x6bd3267 thru 0x6be1e93 (0xec2d bytes) 2024-09-21T23:10:51.700 INFO:teuthology.orchestra.run.smithi150.stdout:2986 writesame 0xa87a910 thru 0xa8813fd data_size 0x1(0x6aee bytes) 2024-09-21T23:10:51.717 INFO:teuthology.orchestra.run.smithi150.stdout:2987 trunc from 0xc727e28 to 0xa90c40c 2024-09-21T23:10:51.742 INFO:teuthology.orchestra.run.smithi150.stdout:2988 write 0xb14d9ea thru 0xb159240 (0xb857 bytes) 2024-09-21T23:10:51.748 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:51.748+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:51.748 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.749 INFO:teuthology.orchestra.run.smithi150.stdout:2991 writesame 0x525f5d5 thru 0x5265751 data_size 0x9(0x617d bytes) 2024-09-21T23:10:51.756 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.756 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:51.756 INFO:teuthology.orchestra.run.smithi150.stdout:2994 read 0xb5fbcb thru 0xb6f1d4 (0xf60a bytes) 2024-09-21T23:10:51.757 INFO:teuthology.orchestra.run.smithi150.stdout:2995 read 0x673e496 thru 0x67423ed (0x3f58 bytes) 2024-09-21T23:10:51.764 INFO:teuthology.orchestra.run.smithi150.stdout:2997 writesame 0xb949e66 thru 0xb94bdd4 data_size 0x1(0x1f6f bytes) 2024-09-21T23:10:51.765 INFO:teuthology.orchestra.run.smithi150.stdout:2999 write 0x356f75f thru 0x3576d38 (0x75da bytes) 2024-09-21T23:10:51.776 INFO:teuthology.orchestra.run.smithi150.stdout:3000 writesame 0xc32d52b thru 0xc3334bc data_size 0xd(0x5f92 bytes) 2024-09-21T23:10:51.776 INFO:teuthology.orchestra.run.smithi150.stdout:3001 write 0x21e86f2 thru 0x21ef70d (0x701c bytes) 2024-09-21T23:10:51.781 INFO:teuthology.orchestra.run.smithi150.stdout:3004 read 0x1f2040c thru 0x1f27371 (0x6f66 bytes) 2024-09-21T23:10:51.781 INFO:teuthology.orchestra.run.smithi150.stdout:3005 trunc from 0xc3334bd to 0xc2cad55 2024-09-21T23:10:51.784 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:51.784 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:51.787 INFO:teuthology.orchestra.run.smithi150.stdout:3006 read 0x8a3e2f5 thru 0x8a46a68 (0x8774 bytes) 2024-09-21T23:10:51.788 INFO:teuthology.orchestra.run.smithi150.stdout:3007 read 0x5f6b157 thru 0x5f6c4bc (0x1366 bytes) 2024-09-21T23:10:51.788 INFO:teuthology.orchestra.run.smithi150.stdout:3008 read 0x380f12c thru 0x381c401 (0xd2d6 bytes) 2024-09-21T23:10:51.798 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:10:51.798 INFO:teuthology.orchestra.run.smithi150.stdout:3009 trunc from 0xc2cad55 to 0x592a11b 2024-09-21T23:10:51.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:51.849 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:51.847+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:51.950 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:51.948+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:51.983 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:51.983 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:51.997 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:52.084 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:52.084+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:52.152 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:52.152+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:52.164 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:52.164 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:52.178 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:52.264 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:52.264+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:52.345 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:52.345 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:52.354 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:52.351+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:52.359 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:52.455 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:52.455+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:52.524 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:52.524 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:52.538 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:52.544 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:52.555 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:52.555+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:52.657 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:52.656+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:52.663 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:52.663 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:52.679 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:52.722 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:52.723 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:52.739 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:52.740 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:52.756 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:52.757 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:52.757 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:52.755+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:52.773 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:52.773 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:52.773 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:52.794 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:52.794 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:52.825 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:52.825 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:52.838 INFO:teuthology.orchestra.run.smithi150.stdout:3010 trunc from 0x592a11b to 0xbb57a27 2024-09-21T23:10:52.846 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:52.846 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:52.846 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:52.846 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:52.847 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:52.847 INFO:teuthology.orchestra.run.smithi150.stdout:3011 trunc from 0xbb57a27 to 0xd82ba6c 2024-09-21T23:10:52.847 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:52.847 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:52.851 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:52.851 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:52.854 INFO:teuthology.orchestra.run.smithi150.stdout:3012 write 0x1125545 thru 0x113448a (0xef46 bytes) 2024-09-21T23:10:52.857 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:52.856+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:52.861 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:52.861 INFO:teuthology.orchestra.run.smithi150.stdout:3015 read 0x441ffe5 thru 0x442b57b (0xb597 bytes) 2024-09-21T23:10:52.861 INFO:teuthology.orchestra.run.smithi150.stdout:3016 punch from 0xafcd40b to 0xafda1e7, (0xcddc bytes) 2024-09-21T23:10:52.865 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:52.870 INFO:teuthology.orchestra.run.smithi150.stdout:3017 trunc from 0xd82ba6c to 0x74754f7 2024-09-21T23:10:52.885 INFO:teuthology.orchestra.run.smithi150.stdout:3018 write 0x7fc367a thru 0x7fc9750 (0x60d7 bytes) 2024-09-21T23:10:52.890 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:52.890 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:52.905 INFO:teuthology.orchestra.run.smithi150.stdout:3023 write 0xe3b15b8 thru 0xe3b75e2 (0x602b bytes) 2024-09-21T23:10:52.909 INFO:teuthology.orchestra.run.smithi150.stdout:3024 write 0x97966a5 thru 0x979e7a6 (0x8102 bytes) 2024-09-21T23:10:52.917 INFO:teuthology.orchestra.run.smithi150.stdout:3025 punch from 0x1438a90 to 0x1439370, (0x8e0 bytes) 2024-09-21T23:10:52.917 INFO:teuthology.orchestra.run.smithi150.stdout:3026 punch from 0xc25d0ef to 0xc2680b4, (0xafc5 bytes) 2024-09-21T23:10:52.917 INFO:teuthology.orchestra.run.smithi150.stdout:3030 read 0x90168b9 thru 0x9024cbd (0xe405 bytes) 2024-09-21T23:10:52.917 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:52.917 INFO:teuthology.orchestra.run.smithi150.stdout:3032 writesame 0x1802504 thru 0x1804d9d data_size 0x1(0x289a bytes) 2024-09-21T23:10:52.924 INFO:teuthology.orchestra.run.smithi150.stdout:3033 write 0x2813287 thru 0x281da80 (0xa7fa bytes) 2024-09-21T23:10:52.930 INFO:teuthology.orchestra.run.smithi150.stdout:3034 write 0xe19a918 thru 0xe19b90b (0xff4 bytes) 2024-09-21T23:10:52.933 INFO:teuthology.orchestra.run.smithi150.stdout:3035 punch from 0xd9c2cc7 to 0xd9d2847, (0xfb80 bytes) 2024-09-21T23:10:52.933 INFO:teuthology.orchestra.run.smithi150.stdout:3036 write 0x236bd2 thru 0x23cb3f (0x5f6e bytes) 2024-09-21T23:10:52.947 INFO:teuthology.orchestra.run.smithi150.stdout:3037 punch from 0xa9bc814 to 0xa9bfd32, (0x351e bytes) 2024-09-21T23:10:52.947 INFO:teuthology.orchestra.run.smithi150.stdout:3038 read 0x913d38e thru 0x9142a6a (0x56dd bytes) 2024-09-21T23:10:52.947 INFO:teuthology.orchestra.run.smithi150.stdout:3039 write 0x32df662 thru 0x32e8ce4 (0x9683 bytes) 2024-09-21T23:10:52.957 INFO:teuthology.orchestra.run.smithi150.stdout:3040 write 0xadfe811 thru 0xae028fd (0x40ed bytes) 2024-09-21T23:10:52.958 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:52.956+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:52.963 INFO:teuthology.orchestra.run.smithi150.stdout:3041 writesame 0xb1fd0b5 thru 0xb1ff0fc data_size 0x4(0x2048 bytes) 2024-09-21T23:10:52.966 INFO:teuthology.orchestra.run.smithi150.stdout:3043 write 0x5fbb15 thru 0x5ff4cc (0x39b8 bytes) 2024-09-21T23:10:52.976 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:52.976 INFO:teuthology.orchestra.run.smithi150.stdout:3046 writesame 0x1740d4b thru 0x17434c2 data_size 0xc(0x2778 bytes) 2024-09-21T23:10:52.981 INFO:teuthology.orchestra.run.smithi150.stdout:3048 punch from 0xbb2bd3b to 0xbb3978f, (0xda54 bytes) 2024-09-21T23:10:52.981 INFO:teuthology.orchestra.run.smithi150.stdout:3049 write 0x959e859 thru 0x95acda8 (0xe550 bytes) 2024-09-21T23:10:52.989 INFO:teuthology.orchestra.run.smithi150.stdout:3050 write 0x655d96d thru 0x656c1b7 (0xe84b bytes) 2024-09-21T23:10:53.004 INFO:teuthology.orchestra.run.smithi150.stdout:3051 trunc from 0xe3b75e3 to 0xa5d1354 2024-09-21T23:10:53.006 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:53.006 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:53.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:53.006 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:53.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5140755779999999, 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:10:53.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5140221399999998, 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:10:53.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:10:53.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:10:53.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5139711649999992, 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:10:53.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5139202340000004, 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5138649229999999, 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:10:53.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:10:53.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5137960079999999, 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5091586550000002, 2024-09-21T23:10:53.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:10:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:10:53.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:10:53.036 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5090905140000004, 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0229370029999991, 2024-09-21T23:10:53.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:10:53.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41320 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9289728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983670+0000", 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0219517390000004, 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019152985000000001, 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41320 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983670+0000", 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983670+0000", 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983672+0000", 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983676+0000", 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3159999999999998e-06 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983688+0000", 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2493e-05 2024-09-21T23:10:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983698+0000", 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0152e-05 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002623+0000", 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018924252999999999 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002701+0000", 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8372000000000001e-05 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002823+0000", 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012227900000000001 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0218836279999994, 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0218372609999999, 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:10:53.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:10:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:10:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0217090819999992, 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:10:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:10:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:10:53.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0216232870000006, 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:10:53.060 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0210794720000003, 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:10:53.064 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:10:53.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:10:53.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0209816919999994, 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0209068180000003, 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:10:53.072 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0208460400000003, 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:10:53.076 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.0208097219999992, 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:10:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.2163479280000002, 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:10:53.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:10:53.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.088 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.089 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:53.089 INFO:teuthology.orchestra.run.smithi150.stdout:3052 read 0x7ef8cba thru 0x7f069b1 (0xdcf8 bytes) 2024-09-21T23:10:53.089 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.090 INFO:teuthology.orchestra.run.smithi150.stdout:3056 read 0x8dfd83d thru 0x8e0aaeb (0xd2af bytes) 2024-09-21T23:10:53.090 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:53.091 INFO:teuthology.orchestra.run.smithi150.stdout:3057 write 0xd704b04 thru 0xd71016e (0xb66b bytes) 2024-09-21T23:10:53.091 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.091 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.091 INFO:teuthology.orchestra.run.smithi150.stdout:3060 trunc from 0xd71016f to 0x20de869 2024-09-21T23:10:53.091 INFO:teuthology.orchestra.run.smithi150.stdout:3064 read 0x125cfac thru 0x1262e81 (0x5ed6 bytes) 2024-09-21T23:10:53.091 INFO:teuthology.orchestra.run.smithi150.stdout:3065 punch from 0x12d58e7 to 0x12e1cd8, (0xc3f1 bytes) 2024-09-21T23:10:53.092 INFO:teuthology.orchestra.run.smithi150.stdout:3066 read 0xe4d17e thru 0xe50700 (0x3583 bytes) 2024-09-21T23:10:53.092 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:53.088+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:53.110 INFO:teuthology.orchestra.run.smithi150.stdout:3067 writesame 0xcd2bf6a thru 0xcd33520 data_size 0x31(0x75b7 bytes) 2024-09-21T23:10:53.110 INFO:teuthology.orchestra.run.smithi150.stdout:3068 punch from 0x5e28246 to 0x5e2fa11, (0x77cb bytes) 2024-09-21T23:10:53.110 INFO:teuthology.orchestra.run.smithi150.stdout:3069 punch from 0x7ec7e32 to 0x7ecb51a, (0x36e8 bytes) 2024-09-21T23:10:53.111 INFO:teuthology.orchestra.run.smithi150.stdout:3070 read 0x40d8d73 thru 0x40e4c37 (0xbec5 bytes) 2024-09-21T23:10:53.111 INFO:teuthology.orchestra.run.smithi150.stdout:3073 read 0x6c28f8 thru 0x6ced0c (0xc415 bytes) 2024-09-21T23:10:53.112 INFO:teuthology.orchestra.run.smithi150.stdout:3074 read 0x982079d thru 0x9823ee2 (0x3746 bytes) 2024-09-21T23:10:53.113 INFO:teuthology.orchestra.run.smithi150.stdout:3076 punch from 0x59091f2 to 0x590c5e1, (0x33ef bytes) 2024-09-21T23:10:53.113 INFO:teuthology.orchestra.run.smithi150.stdout:3078 write 0x6c7cdc9 thru 0x6c8178c (0x49c4 bytes) 2024-09-21T23:10:53.130 INFO:teuthology.orchestra.run.smithi150.stdout:3080 writesame 0xe8b9a29 thru 0xe8c333d data_size 0x1(0x9915 bytes) 2024-09-21T23:10:53.130 INFO:teuthology.orchestra.run.smithi150.stdout:3081 writesame 0x1350155 thru 0x13548ee data_size 0x2f(0x479a bytes) 2024-09-21T23:10:53.135 INFO:teuthology.orchestra.run.smithi150.stdout:3082 write 0x2c5c9c4 thru 0x2c67170 (0xa7ad bytes) 2024-09-21T23:10:53.142 INFO:teuthology.orchestra.run.smithi150.stdout:3083 punch from 0x6ca5e69 to 0x6cab708, (0x589f bytes) 2024-09-21T23:10:53.168 INFO:teuthology.orchestra.run.smithi150.stdout:3085 trunc from 0xe8c333e to 0xfb5608 2024-09-21T23:10:53.191 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:53.188+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:53.203 INFO:teuthology.orchestra.run.smithi150.stdout:3086 writesame 0x7a1dea6 thru 0x7a27ba7 data_size 0x63(0x9d02 bytes) 2024-09-21T23:10:53.219 INFO:teuthology.orchestra.run.smithi150.stdout:3087 write 0xe247485 thru 0xe250de7 (0x9963 bytes) 2024-09-21T23:10:53.226 INFO:teuthology.orchestra.run.smithi150.stdout:3088 write 0xb6c5096 thru 0xb6d484e (0xf7b9 bytes) 2024-09-21T23:10:53.233 INFO:teuthology.orchestra.run.smithi150.stdout:3090 write 0xb571c5 thru 0xb5cc98 (0x5ad4 bytes) 2024-09-21T23:10:53.248 INFO:teuthology.orchestra.run.smithi150.stdout:3091 trunc from 0xe250de8 to 0x5a7caf2 2024-09-21T23:10:53.263 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.263 INFO:teuthology.orchestra.run.smithi150.stdout:3094 read 0x1f53b9b thru 0x1f55cba (0x2120 bytes) 2024-09-21T23:10:53.263 INFO:teuthology.orchestra.run.smithi150.stdout:3095 write 0x405f191 thru 0x4066308 (0x7178 bytes) 2024-09-21T23:10:53.268 INFO:teuthology.orchestra.run.smithi150.stdout:3096 trunc from 0x5a7caf2 to 0x523088a 2024-09-21T23:10:53.274 INFO:teuthology.orchestra.run.smithi150.stdout:3097 writesame 0x463b0fb thru 0x463fcb7 data_size 0x17(0x4bbd bytes) 2024-09-21T23:10:53.278 INFO:teuthology.orchestra.run.smithi150.stdout:3098 read 0x2014133 thru 0x2018120 (0x3fee bytes) 2024-09-21T23:10:53.278 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.278 INFO:teuthology.orchestra.run.smithi150.stdout:3101 write 0x41544f4 thru 0x41638df (0xf3ec bytes) 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:55003 3.b 3:d2c316ff:::rbd_data.1149796cbcee.000000000000010f:head [set-alloc-hint object_size 262144 write_size 262144,write 0~32768 in=32768b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:53.281941+0000", 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00073991400000000004, 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00077258199999999996, 2024-09-21T23:10:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 55003 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.281941+0000", 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.281941+0000", 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.281942+0000", 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.281968+0000", 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6789000000000001e-05 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.281981+0000", 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2401e-05 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.281988+0000", 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282043+0000", 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5423000000000001e-05 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282187+0000", 2024-09-21T23:10:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014353900000000001 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282327+0000", 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014065 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:55002 3.19 e64/21 3:9aee3ee3:::rbd_data.1149796cbcee.000000000000010e:head v 64'743, mlcod=64'741)", 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:53.282238+0000", 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00044255400000000001, 2024-09-21T23:10:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00056270699999999996, 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282238+0000", 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282238+0000", 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282239+0000", 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.314e-06 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282271+0000", 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2039000000000003e-05 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282282+0000", 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0909e-05 2024-09-21T23:10:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282289+0000", 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5660000000000003e-06 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282350+0000", 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1462000000000002e-05 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:53.282372+0000", 2024-09-21T23:10:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2279000000000001e-05 2024-09-21T23:10:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:10:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 2 2024-09-21T23:10:53.290 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:53.292 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:53.292+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:53.294 INFO:teuthology.orchestra.run.smithi150.stdout:3102 trunc from 0x523088a to 0xb7a91a5 2024-09-21T23:10:53.299 INFO:teuthology.orchestra.run.smithi150.stdout:3103 write 0x9b5b506 thru 0x9b690d7 (0xdbd2 bytes) 2024-09-21T23:10:53.304 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:53.307 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.307 INFO:teuthology.orchestra.run.smithi150.stdout:3105 write 0xb7a35b5 thru 0xb7a7df1 (0x483d bytes) 2024-09-21T23:10:53.311 INFO:teuthology.orchestra.run.smithi150.stdout:3106 write 0xa88f4b2 thru 0xa89e2a3 (0xedf2 bytes) 2024-09-21T23:10:53.319 INFO:teuthology.orchestra.run.smithi150.stdout:3107 write 0x6888820 thru 0x6889b79 (0x135a bytes) 2024-09-21T23:10:53.322 INFO:teuthology.orchestra.run.smithi150.stdout:3108 write 0x3e9b4d3 thru 0x3ea87c8 (0xd2f6 bytes) 2024-09-21T23:10:53.329 INFO:teuthology.orchestra.run.smithi150.stdout:3109 write 0x26c4cf thru 0x2792ad (0xcddf bytes) 2024-09-21T23:10:53.349 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.349 INFO:teuthology.orchestra.run.smithi150.stdout:3111 write 0x3c5bb00 thru 0x3c5bbe2 (0xe3 bytes) 2024-09-21T23:10:53.352 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.352 INFO:teuthology.orchestra.run.smithi150.stdout:3114 writesame 0x8412c7f thru 0x841922d data_size 0x1(0x65af bytes) 2024-09-21T23:10:53.380 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.388 INFO:teuthology.orchestra.run.smithi150.stdout:3116 trunc from 0xb7a91a5 to 0x829e7e8 2024-09-21T23:10:53.393 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:53.392+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:53.423 INFO:teuthology.orchestra.run.smithi150.stdout:3119 write 0xa2c0795 thru 0xa2c0ff6 (0x862 bytes) 2024-09-21T23:10:53.427 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.427 INFO:teuthology.orchestra.run.smithi150.stdout:3122 punch from 0xa1f4c4f to 0xa20375b, (0xeb0c bytes) 2024-09-21T23:10:53.427 INFO:teuthology.orchestra.run.smithi150.stdout:3123 write 0x54f75a6 thru 0x5506d9f (0xf7fa bytes) 2024-09-21T23:10:53.435 INFO:teuthology.orchestra.run.smithi150.stdout:3124 writesame 0x56b59f5 thru 0x56b97e4 data_size 0x8(0x3df0 bytes) 2024-09-21T23:10:53.443 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:53.443 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:53.443 INFO:teuthology.orchestra.run.smithi150.stdout:3125 read 0x987a8e2 thru 0x988225b (0x797a bytes) 2024-09-21T23:10:53.445 INFO:teuthology.orchestra.run.smithi150.stdout:3126 trunc from 0xa2c0ff7 to 0x907974e 2024-09-21T23:10:53.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:53.464 INFO:teuthology.orchestra.run.smithi150.stdout:3127 punch from 0x47b5cfd to 0x47b6649, (0x94c bytes) 2024-09-21T23:10:53.464 INFO:teuthology.orchestra.run.smithi150.stdout:3130 read 0x876db2 thru 0x87da5b (0x6caa bytes) 2024-09-21T23:10:53.466 INFO:teuthology.orchestra.run.smithi150.stdout:3131 write 0x3ba5d59 thru 0x3bb3baa (0xde52 bytes) 2024-09-21T23:10:53.476 INFO:teuthology.orchestra.run.smithi150.stdout:3132 writesame 0x84e0cec thru 0x84eaceb data_size 0x14(0xa000 bytes) 2024-09-21T23:10:53.479 INFO:teuthology.orchestra.run.smithi150.stdout:3134 punch from 0x96d123 to 0x9788c9, (0xb7a6 bytes) 2024-09-21T23:10:53.480 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.480 INFO:teuthology.orchestra.run.smithi150.stdout:3136 writesame 0x8480e0d thru 0x84894ba data_size 0x1(0x86ae bytes) 2024-09-21T23:10:53.494 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:53.492+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:53.520 INFO:teuthology.orchestra.run.smithi150.stdout:3138 writesame 0x2824da2 thru 0x28269e9 data_size 0x14(0x1c48 bytes) 2024-09-21T23:10:53.526 INFO:teuthology.orchestra.run.smithi150.stdout:3139 write 0x849584e thru 0x84a08e1 (0xb094 bytes) 2024-09-21T23:10:53.532 INFO:teuthology.orchestra.run.smithi150.stdout:3140 read 0x2689a4c thru 0x2693ec5 (0xa47a bytes) 2024-09-21T23:10:53.532 INFO:teuthology.orchestra.run.smithi150.stdout:3141 writesame 0x29248e2 thru 0x292c733 data_size 0x2e(0x7e52 bytes) 2024-09-21T23:10:53.537 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.537 INFO:teuthology.orchestra.run.smithi150.stdout:3143 read 0x1bee535 thru 0x1bfc272 (0xdd3e bytes) 2024-09-21T23:10:53.545 INFO:teuthology.orchestra.run.smithi150.stdout:3144 trunc from 0x907974e to 0x48385a8 2024-09-21T23:10:53.561 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.561 INFO:teuthology.orchestra.run.smithi150.stdout:3147 punch from 0x1bb997d to 0x1bc34b6, (0x9b39 bytes) 2024-09-21T23:10:53.576 INFO:teuthology.orchestra.run.smithi150.stdout:3148 write 0x91b4b4b thru 0x91b8256 (0x370c bytes) 2024-09-21T23:10:53.582 INFO:teuthology.orchestra.run.smithi150.stdout:3149 writesame 0x1772764 thru 0x177b8c3 data_size 0x10(0x9160 bytes) 2024-09-21T23:10:53.589 INFO:teuthology.orchestra.run.smithi150.stdout:3151 writesame 0x87ec3b2 thru 0x87f6c9e data_size 0x1(0xa8ed bytes) 2024-09-21T23:10:53.595 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:53.595+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:53.604 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:53.604 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.528258823000002, 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:53.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.362491972000001, 2024-09-21T23:10:53.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:53.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:38234 3.1e 3:7a5571f8:::rbd_data.1149ed0ffb75.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.634546818, 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043458748999999998, 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 38234 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968400+0000", 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6360000000000001e-06 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968406+0000", 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.81e-06 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968419+0000", 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3046e-05 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968426+0000", 2024-09-21T23:10:53.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5440000000000004e-06 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968519+0000", 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3348000000000001e-05 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968668+0000", 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148264 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:10:53.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.977244+0000", 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085759600000000005 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011662+0000", 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034417917999999999 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011690+0000", 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8368999999999999e-05 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:53.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011780+0000", 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0450000000000003e-05 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011857+0000", 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6403999999999996e-05 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52271 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:53.621 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2445856329999998, 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039664149000000003, 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52271 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358360+0000", 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358362+0000", 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9860000000000001e-06 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358368+0000", 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358372+0000", 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7859999999999996e-06 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398016+0000", 2024-09-21T23:10:53.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039643954000000002 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398023+0000", 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9249999999999998e-06 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52394 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2443194569999996, 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039669796, 2024-09-21T23:10:53.625 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52394 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358626+0000", 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6800000000000001e-07 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358628+0000", 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:10:53.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358634+0000", 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358639+0000", 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.772e-06 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398288+0000", 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039648640999999998 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398295+0000", 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0720000000000002e-06 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52395 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2442983249999999, 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039678049999999999, 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.629 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52395 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358647+0000", 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358649+0000", 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569999999999998e-06 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358655+0000", 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.127e-06 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358660+0000", 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6820000000000002e-06 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398317+0000", 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039657215000000003 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398325+0000", 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4789999999999999e-06 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52396 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2442783879999997, 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039688093000000001, 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.633 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52396 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358667+0000", 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358669+0000", 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1509999999999998e-06 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358676+0000", 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-06 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358681+0000", 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1420000000000001e-06 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398348+0000", 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039666202999999997 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398355+0000", 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.861e-06 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.636 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52397 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2442524050000001, 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039702474000000001, 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52397 2024-09-21T23:10:53.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358693+0000", 2024-09-21T23:10:53.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358695+0000", 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358701+0000", 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358706+0000", 2024-09-21T23:10:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8550000000000001e-06 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398385+0000", 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039679157999999999 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398395+0000", 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9299999999999998e-06 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52398 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2442310900000004, 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039712074999999999, 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52398 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:53.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358716+0000", 2024-09-21T23:10:53.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0940000000000002e-06 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358722+0000", 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-06 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358727+0000", 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0760000000000002e-06 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398418+0000", 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039691048 2024-09-21T23:10:53.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398426+0000", 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52399 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2442111789999997, 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039729353000000002, 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.644 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52399 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358736+0000", 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358743+0000", 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4119999999999998e-06 2024-09-21T23:10:53.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358747+0000", 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8749999999999999e-06 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398453+0000", 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039705424000000003 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398463+0000", 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0182e-05 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.647 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52432 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2441912009999996, 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039740801999999999, 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52432 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.648 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358756+0000", 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358762+0000", 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358767+0000", 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6129999999999997e-06 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398487+0000", 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039720278999999997 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398494+0000", 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5619999999999998e-06 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52433 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:53.651 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2441706000000003, 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039750774000000003, 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52433 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358775+0000", 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358777+0000", 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358783+0000", 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358788+0000", 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.135e-06 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398518+0000", 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039730389999999997 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398525+0000", 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52434 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2441508309999998, 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039771847999999999, 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.655 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52434 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:53.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358796+0000", 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8899999999999999e-06 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358802+0000", 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7139999999999998e-06 2024-09-21T23:10:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358807+0000", 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8029999999999996e-06 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398556+0000", 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039749022000000002 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398566+0000", 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0067e-05 2024-09-21T23:10:53.658 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52435 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2441312040000003, 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039820863999999997, 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.659 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52435 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8599999999999999e-07 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358816+0000", 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2180000000000002e-06 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358822+0000", 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9939999999999997e-06 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358827+0000", 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7670000000000003e-06 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398624+0000", 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797341 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398634+0000", 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0158e-05 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52499 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2438425520000003, 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039762180000000001, 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52499 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.663 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359103+0000", 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:10:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359105+0000", 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359111+0000", 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1120000000000001e-06 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359116+0000", 2024-09-21T23:10:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398857+0000", 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039741132999999998 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398864+0000", 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5870000000000004e-06 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.666 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52500 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2438226940000003, 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776995000000002, 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52500 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:53.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359123+0000", 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359124+0000", 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9750000000000001e-06 2024-09-21T23:10:53.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359131+0000", 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359135+0000", 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7520000000000004e-06 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398892+0000", 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039756315 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398899+0000", 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.43e-06 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52526 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2438034160000004, 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039791576000000002, 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.670 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52526 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359142+0000", 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359144+0000", 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.838e-06 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359150+0000", 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359154+0000", 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398925+0000", 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039770497000000002 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398933+0000", 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1319999999999994e-06 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52527 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.243784174, 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03980852, 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52527 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359163+0000", 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.995e-06 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359169+0000", 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9710000000000001e-06 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359174+0000", 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0629999999999996e-06 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398961+0000", 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039786948000000003 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398969+0000", 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1580000000000005e-06 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52528 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.243764573, 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039822428999999999, 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52528 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359181+0000", 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359187+0000", 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359192+0000", 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7509999999999997e-06 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359198+0000", 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9830000000000001e-06 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398995+0000", 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797036000000001 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399003+0000", 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.681 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52529 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.2437394319999999, 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039831105999999998, 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52529 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359206+0000", 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359208+0000", 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359214+0000", 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1e-06 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359219+0000", 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7879999999999997e-06 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399029+0000", 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039810509000000001 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399036+0000", 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:53.685 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:53.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:53.687 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.687 INFO:teuthology.orchestra.run.smithi150.stdout:3153 write 0x91c8ca5 thru 0x91d3b1f (0xae7b bytes) 2024-09-21T23:10:53.687 INFO:teuthology.orchestra.run.smithi150.stdout:3154 writesame 0x55173ce thru 0x5524fc3 data_size 0xf(0xdbf6 bytes) 2024-09-21T23:10:53.687 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.688 INFO:teuthology.orchestra.run.smithi150.stdout:3156 writesame 0xb866f80 thru 0xb86b8ed data_size 0x27(0x496e bytes) 2024-09-21T23:10:53.688 INFO:teuthology.orchestra.run.smithi150.stdout:3157 write 0x4d7cec thru 0x4e62f0 (0xe605 bytes) 2024-09-21T23:10:53.689 INFO:teuthology.orchestra.run.smithi150.stdout:3158 trunc from 0xb86b8ee to 0xe439e4d 2024-09-21T23:10:53.696 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:53.696+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:53.698 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.698 INFO:teuthology.orchestra.run.smithi150.stdout:3160 read 0x38988de thru 0x38a1825 (0x8f48 bytes) 2024-09-21T23:10:53.698 INFO:teuthology.orchestra.run.smithi150.stdout:3161 write 0xbd0c065 thru 0xbd1a3f6 (0xe392 bytes) 2024-09-21T23:10:53.709 INFO:teuthology.orchestra.run.smithi150.stdout:3162 write 0x84c2db8 thru 0x84ca9b8 (0x7c01 bytes) 2024-09-21T23:10:53.715 INFO:teuthology.orchestra.run.smithi150.stdout:3164 write 0x838fa85 thru 0x8391f46 (0x24c2 bytes) 2024-09-21T23:10:53.717 INFO:teuthology.orchestra.run.smithi150.stdout:3165 read 0x4ffefa6 thru 0x5007e14 (0x8e6f bytes) 2024-09-21T23:10:53.717 INFO:teuthology.orchestra.run.smithi150.stdout:3168 read 0xc2b3af6 thru 0xc2bc9b9 (0x8ec4 bytes) 2024-09-21T23:10:53.717 INFO:teuthology.orchestra.run.smithi150.stdout:3170 punch from 0x9c03811 to 0x9c09e99, (0x6688 bytes) 2024-09-21T23:10:53.719 INFO:teuthology.orchestra.run.smithi150.stdout:3171 trunc from 0xe439e4d to 0xd503b24 2024-09-21T23:10:53.726 INFO:teuthology.orchestra.run.smithi150.stdout:3172 read 0x708aa6c thru 0x708fc61 (0x51f6 bytes) 2024-09-21T23:10:53.728 INFO:teuthology.orchestra.run.smithi150.stdout:3175 trunc from 0xd503b24 to 0xe0216ba 2024-09-21T23:10:53.745 INFO:teuthology.orchestra.run.smithi150.stdout:3176 trunc from 0xe0216ba to 0xae387d9 2024-09-21T23:10:53.774 INFO:teuthology.orchestra.run.smithi150.stdout:3178 trunc from 0xae387d9 to 0x4f81756 2024-09-21T23:10:53.796 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.797 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:53.795+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:53.818 INFO:teuthology.orchestra.run.smithi150.stdout:3186 writesame 0x98ad88a thru 0x98b4d4d data_size 0x5e(0x74c4 bytes) 2024-09-21T23:10:53.818 INFO:teuthology.orchestra.run.smithi150.stdout:3188 read 0x6a1490a thru 0x6a1724d (0x2944 bytes) 2024-09-21T23:10:53.818 INFO:teuthology.orchestra.run.smithi150.stdout:3189 punch from 0x30bca1d to 0x30cb2ad, (0xe890 bytes) 2024-09-21T23:10:53.818 INFO:teuthology.orchestra.run.smithi150.stdout:3191 read 0x662f5b4 thru 0x663e848 (0xf295 bytes) 2024-09-21T23:10:53.818 INFO:teuthology.orchestra.run.smithi150.stdout:3192 read 0x457dadc thru 0x4586e48 (0x936d bytes) 2024-09-21T23:10:53.819 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:53.819 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:53.822 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:10:53.823 INFO:teuthology.orchestra.run.smithi150.stdout:3193 trunc from 0x98b4d4e to 0x79b67e4 2024-09-21T23:10:53.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:53.840 INFO:teuthology.orchestra.run.smithi150.stdout:3197 trunc from 0x79b67e4 to 0xadd0de1 2024-09-21T23:10:53.848 INFO:teuthology.orchestra.run.smithi150.stdout:3198 write 0x47a8049 thru 0x47b42bf (0xc277 bytes) 2024-09-21T23:10:53.855 INFO:teuthology.orchestra.run.smithi150.stdout:3199 read 0x31ebf57 thru 0x31ed5ee (0x1698 bytes) 2024-09-21T23:10:53.855 INFO:teuthology.orchestra.run.smithi150.stdout:3200 write 0x7ff044a thru 0x7ff8626 (0x81dd bytes) 2024-09-21T23:10:53.860 INFO:teuthology.orchestra.run.smithi150.stdout:3201 write 0x512cd62 thru 0x51326b4 (0x5953 bytes) 2024-09-21T23:10:53.865 INFO:teuthology.orchestra.run.smithi150.stdout:3202 read 0xa2bd9bf thru 0xa2c6cf4 (0x9336 bytes) 2024-09-21T23:10:53.865 INFO:teuthology.orchestra.run.smithi150.stdout:3203 read 0x335ef9e thru 0x3364fd2 (0x6035 bytes) 2024-09-21T23:10:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:3206 writesame 0x6b7c717 thru 0x6b8be80 data_size 0xb(0xf76a bytes) 2024-09-21T23:10:53.874 INFO:teuthology.orchestra.run.smithi150.stdout:3207 write 0x907a1c thru 0x909d2c (0x2311 bytes) 2024-09-21T23:10:53.888 INFO:teuthology.orchestra.run.smithi150.stdout:3208 read 0x14b781a thru 0x14bb2e9 (0x3ad0 bytes) 2024-09-21T23:10:53.888 INFO:teuthology.orchestra.run.smithi150.stdout:3210 read 0x9a48ff0 thru 0x9a51914 (0x8925 bytes) 2024-09-21T23:10:53.888 INFO:teuthology.orchestra.run.smithi150.stdout:3214 read 0x36a3e6b thru 0x36a7fd5 (0x416b bytes) 2024-09-21T23:10:53.888 INFO:teuthology.orchestra.run.smithi150.stdout:3215 trunc from 0xadd0de1 to 0xb3c0fc0 2024-09-21T23:10:53.896 INFO:teuthology.orchestra.run.smithi150.stdout:3216 punch from 0x8cb32ce to 0x8cbb725, (0x8457 bytes) 2024-09-21T23:10:53.896 INFO:teuthology.orchestra.run.smithi150.stdout:3218 writesame 0x690119 thru 0x69eb99 data_size 0x1(0xea81 bytes) 2024-09-21T23:10:53.901 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T23:10:53.901 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:10:53.901 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-21T23:10:53.901 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T23:10:53.901 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T23:10:53.918 INFO:teuthology.orchestra.run.smithi150.stdout:3220 punch from 0x355ef5c to 0x356a483, (0xb527 bytes) 2024-09-21T23:10:53.918 INFO:teuthology.orchestra.run.smithi150.stdout:3221 read 0x4b5d439 thru 0x4b62138 (0x4d00 bytes) 2024-09-21T23:10:53.919 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:53.919+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:53.920 INFO:tasks.ceph.osd.4:Started 2024-09-21T23:10:53.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:10:53.935 INFO:teuthology.orchestra.run.smithi150.stdout:3223 trunc from 0xb3c0fc0 to 0x284b0f 2024-09-21T23:10:53.998 INFO:teuthology.orchestra.run.smithi150.stdout:3224 trunc from 0x284b0f to 0x3dbe01e 2024-09-21T23:10:54.010 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:53.995+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:54.010 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:54.011 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:54.011 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:54.011 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:54.011 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:54.011 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:54.017 INFO:teuthology.orchestra.run.smithi150.stdout:3226 write 0x85a6064 thru 0x85ab940 (0x58dd bytes) 2024-09-21T23:10:54.021 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:54.021 INFO:teuthology.orchestra.run.smithi150.stdout:3228 read 0x6b63651 thru 0x6b6c4da (0x8e8a bytes) 2024-09-21T23:10:54.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:54.030 INFO:teuthology.orchestra.run.smithi150.stdout:3229 write 0xad10167 thru 0xad15d84 (0x5c1e bytes) 2024-09-21T23:10:54.034 INFO:teuthology.orchestra.run.smithi150.stdout:3230 writesame 0x3df5ee3 thru 0x3df8cee data_size 0x7(0x2e0c bytes) 2024-09-21T23:10:54.041 INFO:teuthology.orchestra.run.smithi150.stdout:3231 read 0x3977246 thru 0x3983206 (0xbfc1 bytes) 2024-09-21T23:10:54.041 INFO:teuthology.orchestra.run.smithi150.stdout:3232 writesame 0xa201f45 thru 0xa20ab29 data_size 0x1(0x8be5 bytes) 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:no valid command found; 10 closest matches: 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:0 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:1 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:2 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:abort 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:assert 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:config diff 2024-09-21T23:10:54.044 INFO:teuthology.orchestra.run.smithi175.stderr:config diff get 2024-09-21T23:10:54.045 INFO:teuthology.orchestra.run.smithi175.stderr:config get 2024-09-21T23:10:54.045 INFO:teuthology.orchestra.run.smithi175.stderr:config help [] 2024-09-21T23:10:54.045 INFO:teuthology.orchestra.run.smithi175.stderr:config set ... 2024-09-21T23:10:54.045 INFO:teuthology.orchestra.run.smithi175.stderr:admin_socket: invalid command 2024-09-21T23:10:54.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:10:54.047 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-21T23:10:54.055 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:54.068 INFO:teuthology.orchestra.run.smithi150.stdout:3239 write 0xed6b1e0 thru 0xed6d5b2 (0x23d3 bytes) 2024-09-21T23:10:54.072 INFO:teuthology.orchestra.run.smithi150.stdout:3240 write 0xdfa365d thru 0xdfa42dc (0xc80 bytes) 2024-09-21T23:10:54.086 INFO:teuthology.orchestra.run.smithi150.stdout:3241 trunc from 0xed6d5b3 to 0x84fd99d 2024-09-21T23:10:54.099 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:54.099+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:54.100 INFO:teuthology.orchestra.run.smithi150.stdout:3244 read 0x72b717c thru 0x72be23d (0x70c2 bytes) 2024-09-21T23:10:54.118 INFO:teuthology.orchestra.run.smithi150.stdout:3245 write 0xedaa687 thru 0xedb3e18 (0x9792 bytes) 2024-09-21T23:10:54.124 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:54.124 INFO:teuthology.orchestra.run.smithi150.stdout:3247 writesame 0x5f56191 thru 0x5f58d08 data_size 0x1a(0x2b78 bytes) 2024-09-21T23:10:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:3248 read 0xa8a43b8 thru 0xa8b0909 (0xc552 bytes) 2024-09-21T23:10:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:3250 write 0x949b4c3 thru 0x94a0ec9 (0x5a07 bytes) 2024-09-21T23:10:54.137 INFO:teuthology.orchestra.run.smithi150.stdout:3251 read 0x61bbff4 thru 0x61c8ca3 (0xccb0 bytes) 2024-09-21T23:10:54.137 INFO:teuthology.orchestra.run.smithi150.stdout:3252 read 0xb6dd60f thru 0xb6df16d (0x1b5f bytes) 2024-09-21T23:10:54.137 INFO:teuthology.orchestra.run.smithi150.stdout:3253 writesame 0x52e5ede thru 0x52e7acd data_size 0x18(0x1bf0 bytes) 2024-09-21T23:10:54.156 INFO:teuthology.orchestra.run.smithi150.stdout:3256 trunc from 0xedb3e19 to 0x620615a 2024-09-21T23:10:54.162 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:54.162 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:54.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:54.162 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:54.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.162 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5124 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.974582888999997, 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046086192999999998, 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5124 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186881+0000", 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.133e-06 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186887+0000", 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7939999999999999e-06 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186904+0000", 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7193e-05 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186918+0000", 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223027+0000", 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036108460000000002 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223201+0000", 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017483900000000001 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.232966+0000", 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097649039999999996 2024-09-21T23:10:54.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5279 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.973304249000002, 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045406334999999999, 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.167 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5279 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188160+0000", 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188167+0000", 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0700000000000001e-06 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188188+0000", 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-05 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188205+0000", 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7444000000000001e-05 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233051+0000", 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044846496999999999 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233167+0000", 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115307 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233565+0000", 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039829399999999999 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5280 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.973235053000003, 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047857909999999997, 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.171 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5280 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:54.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188229+0000", 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.237e-06 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188235+0000", 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188254+0000", 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9015999999999999e-05 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188268+0000", 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.395e-05 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233608+0000", 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045339957 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233722+0000", 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011403700000000001 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236086+0000", 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00236359 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5281 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:54.175 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.973173553000002, 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048399091999999998, 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5281 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:54.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188291+0000", 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188298+0000", 2024-09-21T23:10:54.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188315+0000", 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7819999999999999e-05 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188329+0000", 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3383e-05 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236130+0000", 2024-09-21T23:10:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047800827999999997 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236240+0000", 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110745 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236689+0000", 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044823600000000002 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5282 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.973115673999999, 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048602487, 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5282 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188348+0000", 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188355+0000", 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3029999999999996e-06 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188373+0000", 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8056000000000001e-05 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188386+0000", 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3536999999999999e-05 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236748+0000", 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048361617000000003 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236840+0000", 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2075000000000002e-05 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236950+0000", 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109755 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5283 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.972972415000001, 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048844519000000003, 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5283 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.184 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188493+0000", 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3800000000000001e-06 2024-09-21T23:10:54.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188503+0000", 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188526+0000", 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.304e-05 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188543+0000", 2024-09-21T23:10:54.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7557000000000001e-05 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237020+0000", 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048476656 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237187+0000", 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016670199999999999 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237335+0000", 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014857100000000001 2024-09-21T23:10:54.187 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5284 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.972890548999999, 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049117881000000002, 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.188 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5284 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188574+0000", 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188580+0000", 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188598+0000", 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8304e-05 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188611+0000", 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3329000000000001e-05 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237402+0000", 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048790838000000003 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237549+0000", 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146897 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237690+0000", 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141383 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5285 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.972833262999998, 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049392706000000001, 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.192 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5285 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188631+0000", 2024-09-21T23:10:54.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0400000000000005e-07 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188636+0000", 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8329999999999999e-06 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188655+0000", 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8246000000000001e-05 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188670+0000", 2024-09-21T23:10:54.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5440000000000001e-05 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237748+0000", 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049077975000000003 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237890+0000", 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141953 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238022+0000", 2024-09-21T23:10:54.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013235499999999999 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5286 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.97277416, 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049655168, 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.196 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5286 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188690+0000", 2024-09-21T23:10:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1769999999999999e-06 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188696+0000", 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5269999999999998e-06 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188713+0000", 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7836e-05 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188726+0000", 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:10:54.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238077+0000", 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049350964999999997 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238228+0000", 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015027000000000001 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238344+0000", 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116403 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.199 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5287 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.972004572000003, 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049431115999999997, 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5287 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.200 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189460+0000", 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189470+0000", 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189491+0000", 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0743000000000001e-05 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189503+0000", 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2787e-05 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238395+0000", 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048891284 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238519+0000", 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124745 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238890+0000", 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037020400000000001 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5465 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.964208579999998, 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045516793999999999, 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5465 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197256+0000", 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197262+0000", 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-21T23:10:54.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197280+0000", 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197294+0000", 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3983e-05 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242417+0000", 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045122664999999999 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242550+0000", 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013329499999999999 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242771+0000", 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022120399999999999 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5466 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:54.207 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.964150523000001, 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045852049999999998, 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5466 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197313+0000", 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197321+0000", 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:10:54.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197338+0000", 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7824000000000001e-05 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197353+0000", 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.456e-05 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242853+0000", 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045500119999999998 2024-09-21T23:10:54.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243010+0000", 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015682599999999999 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243165+0000", 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015463400000000001 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5467 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.211 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.964091279999998, 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046123232, 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5467 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197373+0000", 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197378+0000", 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197397+0000", 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8793000000000001e-05 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197411+0000", 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3726999999999999e-05 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243249+0000", 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045838320000000002 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243376+0000", 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012760199999999999 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243495+0000", 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011848100000000001 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5639 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.961279669, 2024-09-21T23:10:54.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045343482999999997, 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5639 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200184+0000", 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200190+0000", 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200209+0000", 2024-09-21T23:10:54.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8563000000000002e-05 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200223+0000", 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4173e-05 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.244967+0000", 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044743567999999997 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245103+0000", 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013634300000000001 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245527+0000", 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042370199999999998 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5640 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.961183812000002, 2024-09-21T23:10:54.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045524901, 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5640 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200280+0000", 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200287+0000", 2024-09-21T23:10:54.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200307+0000", 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0129999999999999e-05 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200321+0000", 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245576+0000", 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045255521 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245701+0000", 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012480100000000001 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245804+0000", 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102933 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5641 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.961121173000002, 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045750444000000001, 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5641 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200343+0000", 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200351+0000", 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9160000000000001e-06 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200369+0000", 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8490000000000001e-05 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200383+0000", 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3234000000000001e-05 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245850+0000", 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045467883000000001 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245979+0000", 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128641 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246092+0000", 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113143 2024-09-21T23:10:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5642 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.96106107, 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045975347999999999, 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5642 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200403+0000", 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200409+0000", 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6570000000000002e-06 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200427+0000", 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8306000000000001e-05 2024-09-21T23:10:54.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200441+0000", 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4042999999999999e-05 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246144+0000", 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045702495000000003 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246277+0000", 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133831 2024-09-21T23:10:54.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246377+0000", 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9835999999999998e-05 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5643 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.961003544, 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046409156, 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.232 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5643 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:54.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200460+0000", 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200466+0000", 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200483+0000", 2024-09-21T23:10:54.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7776999999999999e-05 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200500+0000", 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6852999999999999e-05 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246582+0000", 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046081471999999998 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246725+0000", 2024-09-21T23:10:54.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143184 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246869+0000", 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014361699999999999 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5644 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.960943471, 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046601900000000002, 2024-09-21T23:10:54.236 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5644 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200521+0000", 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200526+0000", 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-06 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200544+0000", 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7410000000000001e-05 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200559+0000", 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5435999999999999e-05 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246920+0000", 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046360902000000002 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247055+0000", 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468200000000001 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247121+0000", 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6712999999999996e-05 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.240 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5645 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.960884745000001, 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.04674565, 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5645 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:54.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200579+0000", 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2270000000000001e-06 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200585+0000", 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:54.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200605+0000", 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9675999999999999e-05 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200618+0000", 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509999999999999e-05 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247148+0000", 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046529451999999999 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247219+0000", 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0833000000000004e-05 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247324+0000", 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105122 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.244 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:54.246 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3257 write 0x8b14131 thru 0x8b18afc (0x49cc bytes) 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3258 write 0x9c7c9c6 thru 0x9c83a9f (0x70da bytes) 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3259 trunc from 0x9c83aa0 to 0x9c33283 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3261 punch from 0x61b1d14 to 0x61ba9f9, (0x8ce5 bytes) 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3262 punch from 0x4ef5dd9 to 0x4f030bb, (0xd2e2 bytes) 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3264 punch from 0x72b4e2d to 0x72b54a0, (0x673 bytes) 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3265 read 0x955e5ff thru 0x955f012 (0xa14 bytes) 2024-09-21T23:10:54.247 INFO:teuthology.orchestra.run.smithi150.stdout:3266 read 0x61b3e2a thru 0x61bd86e (0x9a45 bytes) 2024-09-21T23:10:54.248 INFO:teuthology.orchestra.run.smithi150.stdout:3267 write 0x8eaac9c thru 0x8eb7c03 (0xcf68 bytes) 2024-09-21T23:10:54.248 INFO:teuthology.orchestra.run.smithi150.stdout:3268 read 0x1b9a7cd thru 0x1ba90e1 (0xe915 bytes) 2024-09-21T23:10:54.248 INFO:teuthology.orchestra.run.smithi150.stdout:3270 read 0xa2fbed thru 0xa3aadf (0xaef3 bytes) 2024-09-21T23:10:54.248 INFO:teuthology.orchestra.run.smithi150.stdout:3271 write 0xa4f3449 thru 0xa4f7663 (0x421b bytes) 2024-09-21T23:10:54.248 INFO:teuthology.orchestra.run.smithi150.stdout:3272 read 0x15af9dc thru 0x15b4844 (0x4e69 bytes) 2024-09-21T23:10:54.248 INFO:teuthology.orchestra.run.smithi150.stdout:3274 write 0x176ec8 thru 0x1848de (0xda17 bytes) 2024-09-21T23:10:54.248 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:54.244+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:54.248 INFO:teuthology.orchestra.run.smithi150.stdout:3275 writesame 0x6b8726a thru 0x6b88bf8 data_size 0x3(0x198f bytes) 2024-09-21T23:10:54.257 INFO:teuthology.orchestra.run.smithi150.stdout:3276 punch from 0x20521af to 0x205facd, (0xd91e bytes) 2024-09-21T23:10:54.263 INFO:teuthology.orchestra.run.smithi150.stdout:3277 trunc from 0xa4f7664 to 0xdfac3c5 2024-09-21T23:10:54.273 INFO:teuthology.orchestra.run.smithi150.stdout:3279 punch from 0x1f8d335 to 0x1f90583, (0x324e bytes) 2024-09-21T23:10:54.273 INFO:teuthology.orchestra.run.smithi150.stdout:3280 punch from 0xaa38ad5 to 0xaa4540b, (0xc936 bytes) 2024-09-21T23:10:54.273 INFO:teuthology.orchestra.run.smithi150.stdout:3282 write 0x3010090 thru 0x301496e (0x48df bytes) 2024-09-21T23:10:54.279 INFO:teuthology.orchestra.run.smithi150.stdout:3283 read 0x73ef8f thru 0x74c2ce (0xd340 bytes) 2024-09-21T23:10:54.283 INFO:teuthology.orchestra.run.smithi150.stdout:3287 trunc from 0xdfac3c5 to 0xb72ce7d 2024-09-21T23:10:54.294 INFO:teuthology.orchestra.run.smithi150.stdout:3289 write 0x7a1053a thru 0x7a13f1f (0x39e6 bytes) 2024-09-21T23:10:54.299 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:10:54.300 INFO:teuthology.orchestra.run.smithi150.stdout:3290 write 0xe74e7c thru 0xe77c1c (0x2da1 bytes) 2024-09-21T23:10:54.313 INFO:teuthology.orchestra.run.smithi150.stdout:3292 trunc from 0xb72ce7d to 0xcc11d35 2024-09-21T23:10:54.319 INFO:teuthology.orchestra.run.smithi150.stdout:3293 clone 27 order 22 su 32768 sc 7 2024-09-21T23:10:54.368 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:54.368+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:54.413 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:54.413 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:54.414 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:54.414 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:54.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:54.471 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:54.471+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:54.548 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:54.547+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:54.551 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:54.551 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:54.551 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:54.551 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:54.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:54.649 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:54.648+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:54.729 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:54.729 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:54.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:54.729 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:54.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.729 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:27644 3.10 3:0811e7db:::rbd_data.11499b8b109.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.373258053000001, 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041757458999999997, 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 27644 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353815+0000", 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.426e-06 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359473+0000", 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056580709999999998 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359492+0000", 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8978999999999999e-05 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359500+0000", 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0909999999999993e-06 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359556+0000", 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6317999999999998e-05 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359660+0000", 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103915 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.360026+0000", 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036547400000000002 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.370944+0000", 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010918323000000001 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395472+0000", 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024528372999999999 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395527+0000", 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4873999999999997e-05 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395571+0000", 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3615000000000002e-05 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.999768875000001, 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.735 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:10:54.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:10:54.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:10:54.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39778 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8396800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.500689+0000", 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.226382277000001, 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030452587, 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39778 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500689+0000", 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500689+0000", 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500690+0000", 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1960000000000001e-06 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500696+0000", 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6969999999999998e-06 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500715+0000", 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372e-05 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500729+0000", 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3959e-05 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.530824+0000", 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030095093 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.530940+0000", 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115989 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531142+0000", 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020228100000000001 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.745 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39779 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8511488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.500749+0000", 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.226323067999999, 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030774979000000001, 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39779 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.746 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500749+0000", 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500749+0000", 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500750+0000", 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.164e-06 2024-09-21T23:10:54.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500755+0000", 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.293e-06 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500772+0000", 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7452000000000002e-05 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500786+0000", 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3232e-05 2024-09-21T23:10:54.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531215+0000", 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030429540000000001 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531326+0000", 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110703 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531523+0000", 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019759500000000001 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.749 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39780 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8454144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.500820+0000", 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.226251139, 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031111728000000002, 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39780 2024-09-21T23:10:54.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500820+0000", 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500820+0000", 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500822+0000", 2024-09-21T23:10:54.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1570000000000001e-06 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500840+0000", 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8742000000000001e-05 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500859+0000", 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8989000000000001e-05 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500874+0000", 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4226000000000001e-05 2024-09-21T23:10:54.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531594+0000", 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030720254999999998 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531715+0000", 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012127499999999999 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531932+0000", 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021708400000000001 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.753 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39791 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8544256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.226002017000001, 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031313771999999997, 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39791 2024-09-21T23:10:54.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7000000000000003e-07 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501074+0000", 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501087+0000", 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3115999999999999e-05 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501094+0000", 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4760000000000001e-06 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532017+0000", 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030923302999999999 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532167+0000", 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000149424 2024-09-21T23:10:54.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532383+0000", 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021645 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39792 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8601600~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.225967005999999, 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031745609000000001, 2024-09-21T23:10:54.758 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39792 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9400000000000004e-07 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501109+0000", 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501118+0000", 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7589999999999992e-06 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501126+0000", 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0630000000000006e-06 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532468+0000", 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031341130000000002 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532605+0000", 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013782200000000001 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532850+0000", 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024479800000000001 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39800 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8011776~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.762 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501231+0000", 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.225840324, 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032088402000000002, 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39800 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501231+0000", 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501231+0000", 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501232+0000", 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501235+0000", 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0860000000000002e-06 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501245+0000", 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7059999999999999e-06 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501252+0000", 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9e-06 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532953+0000", 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031701288000000001 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533105+0000", 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000151705 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533320+0000", 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021504299999999999 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.766 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39801 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8069120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.225806876, 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032634899000000002, 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39801 2024-09-21T23:10:54.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:54.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501268+0000", 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9670000000000001e-06 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501281+0000", 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2704999999999999e-05 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501288+0000", 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.114e-06 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533394+0000", 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032106226000000002 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533510+0000", 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001159 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533900+0000", 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038933000000000001 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39814 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8159232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.225699819000001, 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033410207999999997, 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39814 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1999999999999999e-07 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501376+0000", 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.146e-06 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501385+0000", 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7100000000000002e-06 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501393+0000", 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2030000000000003e-06 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534062+0000", 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032669863 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534352+0000", 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028966199999999999 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534782+0000", 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000429904 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39815 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8216576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501402+0000", 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.225669087, 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033993651999999999, 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.775 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39815 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501402+0000", 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501402+0000", 2024-09-21T23:10:54.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501403+0000", 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501407+0000", 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.189e-06 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501424+0000", 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6422999999999998e-05 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501450+0000", 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5962e-05 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534913+0000", 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033463360999999997 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535138+0000", 2024-09-21T23:10:54.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000224849 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535396+0000", 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025820800000000001 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39828 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8306688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501642+0000", 2024-09-21T23:10:54.779 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.225429169, 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034518844, 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39828 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501642+0000", 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501642+0000", 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501644+0000", 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501649+0000", 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.524e-06 2024-09-21T23:10:54.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501667+0000", 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7682000000000001e-05 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501680+0000", 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2962000000000001e-05 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535508+0000", 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033828035999999999 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535723+0000", 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000215267 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536161+0000", 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043817299999999998 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39829 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8364032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501698+0000", 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 11.225373432, 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034896215000000001, 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39829 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.784 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501698+0000", 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501698+0000", 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501699+0000", 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.054e-06 2024-09-21T23:10:54.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501705+0000", 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6550000000000001e-06 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501722+0000", 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7393000000000001e-05 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501735+0000", 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3189e-05 2024-09-21T23:10:54.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536223+0000", 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034487732 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536365+0000", 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142282 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536594+0000", 2024-09-21T23:10:54.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022891 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41982 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 3956736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.630432925000001, 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030321332999999999, 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41982 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096644+0000", 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4669999999999998e-06 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096661+0000", 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6558999999999998e-05 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096671+0000", 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0689e-05 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.126521+0000", 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.029849496 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.126681+0000", 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000160438 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.126960+0000", 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027883399999999999 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41983 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4014080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096695+0000", 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.630376095000001, 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030781096000000001, 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.792 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41983 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096695+0000", 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096695+0000", 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096696+0000", 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2600000000000001e-07 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096701+0000", 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5859999999999998e-06 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096716+0000", 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4901e-05 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096727+0000", 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1551e-05 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127021+0000", 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030293433000000002 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127181+0000", 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016025100000000001 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127477+0000", 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029544800000000002 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41996 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4104192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096862+0000", 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.630209398, 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031125463999999999, 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.796 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41996 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096862+0000", 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096862+0000", 2024-09-21T23:10:54.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096863+0000", 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096868+0000", 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1760000000000001e-06 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096883+0000", 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4938e-05 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096894+0000", 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0329999999999999e-05 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127539+0000", 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030645596000000001 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127701+0000", 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016159099999999999 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127988+0000", 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028668399999999999 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41997 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4161536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096911+0000", 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.630160126, 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.03158888, 2024-09-21T23:10:54.800 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41997 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096911+0000", 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096911+0000", 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096912+0000", 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096917+0000", 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0540000000000002e-06 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096934+0000", 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6070999999999999e-05 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096957+0000", 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3924e-05 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128048+0000", 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031090824999999999 2024-09-21T23:10:54.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128212+0000", 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016409199999999999 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128500+0000", 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028793900000000002 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:42009 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4194304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.804 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.097094+0000", 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.629977976999999, 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031926768000000001, 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 42009 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097094+0000", 2024-09-21T23:10:54.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097094+0000", 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097095+0000", 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.17e-06 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097100+0000", 2024-09-21T23:10:54.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.485e-06 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097116+0000", 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5559000000000001e-05 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097128+0000", 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1786e-05 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128563+0000", 2024-09-21T23:10:54.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031434974999999997 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128722+0000", 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015934099999999999 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129020+0000", 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029845199999999999 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:42010 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4251648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.097146+0000", 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.629925813, 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032297840000000001, 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 42010 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097146+0000", 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097146+0000", 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097147+0000", 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3099999999999996e-07 2024-09-21T23:10:54.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097163+0000", 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5835999999999999e-05 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097177+0000", 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4681999999999999e-05 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097185+0000", 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8140000000000001e-06 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129106+0000", 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031920945999999999 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129228+0000", 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012197399999999999 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129444+0000", 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021565700000000001 2024-09-21T23:10:54.812 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:42012 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4308992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.097223+0000", 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 10.629848752999999, 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032712365, 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:54.813 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 42012 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097223+0000", 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097223+0000", 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:54.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097224+0000", 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2299999999999999e-07 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097228+0000", 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.527e-06 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097240+0000", 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1605e-05 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:54.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097247+0000", 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129498+0000", 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032251552000000003 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129654+0000", 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015591499999999999 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:54.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129935+0000", 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028121100000000002 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:54.817 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:54.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:54.819 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:54.816+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:54.852 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:54.920 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:54.916+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:54.944 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:54.944 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:54.944 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:54.944 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:54.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:55.004 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:55.004 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:55.021 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:55.019+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:55.021 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:55.021 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:55.037 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:55.037 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:55.053 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:55.053 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:55.054 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:55.074 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:55.074 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:55.080 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:55.080 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:55.080 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:55.080 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:55.080 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:55.093 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:55.094 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:55.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:55.111 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:55.111 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:55.115 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:55.116 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:55.139 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:55.139+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:55.222 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:55.219+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:55.259 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:55.259 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:55.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:55.323 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:55.320+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:55.415 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:55.415 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:55.425 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:55.423+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:55.432 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:55.501 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone26 from 0xcc11d35 (overlap 0x284b0f) to 0xc1684 2024-09-21T23:10:55.525 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:55.524+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:55.623 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:55.623 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:55.626 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:55.624+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:55.639 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:55.701 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-21T23:10:55.727 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:55.724+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:55.801 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:10:55.799+0000 7fdddc67c7c0 -1 Falling back to public interface 2024-09-21T23:10:55.829 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:55.828+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:55.867 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:55.867 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:55.882 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:55.929 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:55.927+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:56.024 INFO:teuthology.orchestra.run.smithi150.stdout:3294 write 0x2a9c5b6 thru 0x2aaac37 (0xe682 bytes) 2024-09-21T23:10:56.030 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:56.027+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:56.043 INFO:teuthology.orchestra.run.smithi150.stdout:3295 read 0x4faa9bc thru 0x4fb9c0f (0xf254 bytes) 2024-09-21T23:10:56.044 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:56.044 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:56.056 INFO:teuthology.orchestra.run.smithi150.stdout:3296 write 0xce94273 thru 0xce96a46 (0x27d4 bytes) 2024-09-21T23:10:56.059 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:56.063 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:56.063 INFO:teuthology.orchestra.run.smithi150.stdout:3299 write 0xb006dd3 thru 0xb00dfec (0x721a bytes) 2024-09-21T23:10:56.070 INFO:teuthology.orchestra.run.smithi150.stdout:3301 writesame 0xb149a49 thru 0xb14b1ec data_size 0x22(0x17a4 bytes) 2024-09-21T23:10:56.075 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:56.075 INFO:teuthology.orchestra.run.smithi150.stdout:3303 write 0x3080fc0 thru 0x30869a0 (0x59e1 bytes) 2024-09-21T23:10:56.077 INFO:teuthology.orchestra.run.smithi150.stdout:3304 read 0x1ed843 thru 0x1f1c86 (0x4444 bytes) 2024-09-21T23:10:56.079 INFO:teuthology.orchestra.run.smithi150.stdout:3305 writesame 0xb388eb1 thru 0xb39559d data_size 0x69(0xc6ed bytes) 2024-09-21T23:10:56.086 INFO:teuthology.orchestra.run.smithi150.stdout:3307 write 0x51644ca thru 0x51728e5 (0xe41c bytes) 2024-09-21T23:10:56.096 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:56.096 INFO:teuthology.orchestra.run.smithi150.stdout:3310 write 0x1a5514 thru 0x1b2ae1 (0xd5ce bytes) 2024-09-21T23:10:56.109 INFO:teuthology.orchestra.run.smithi150.stdout:3311 punch from 0x5006d3c to 0x50137af, (0xca73 bytes) 2024-09-21T23:10:56.109 INFO:teuthology.orchestra.run.smithi150.stdout:3312 write 0xf3307c thru 0xf3e771 (0xb6f6 bytes) 2024-09-21T23:10:56.111 INFO:teuthology.orchestra.run.smithi150.stdout:3313 writesame 0x201b264 thru 0x2026e49 data_size 0x3(0xbbe6 bytes) 2024-09-21T23:10:56.122 INFO:teuthology.orchestra.run.smithi150.stdout:3314 punch from 0x3a2f4ca to 0x3a3598f, (0x64c5 bytes) 2024-09-21T23:10:56.123 INFO:teuthology.orchestra.run.smithi150.stdout:3316 read 0x49ab0b thru 0x4a8251 (0xd747 bytes) 2024-09-21T23:10:56.123 INFO:teuthology.orchestra.run.smithi150.stdout:3318 read 0x84686be thru 0x8478191 (0xfad4 bytes) 2024-09-21T23:10:56.125 INFO:teuthology.orchestra.run.smithi150.stdout:3319 writesame 0xc6bc5ea thru 0xc6be7a2 data_size 0x1(0x21b9 bytes) 2024-09-21T23:10:56.131 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:56.131+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:56.134 INFO:teuthology.orchestra.run.smithi150.stdout:3321 trunc from 0xce96a47 to 0xb1e7c49 2024-09-21T23:10:56.160 INFO:teuthology.orchestra.run.smithi150.stdout:3322 write 0x7893853 thru 0x78a2ecf (0xf67d bytes) 2024-09-21T23:10:56.172 INFO:teuthology.orchestra.run.smithi150.stdout:3323 write 0x70a0c3c thru 0x70adaba (0xce7f bytes) 2024-09-21T23:10:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:3324 read 0x8043057 thru 0x804bc31 (0x8bdb bytes) 2024-09-21T23:10:56.179 INFO:teuthology.orchestra.run.smithi150.stdout:3325 write 0x5b0bbac thru 0x5b136da (0x7b2f bytes) 2024-09-21T23:10:56.188 INFO:teuthology.orchestra.run.smithi150.stdout:3326 write 0x8e0d97a thru 0x8e143b4 (0x6a3b bytes) 2024-09-21T23:10:56.199 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:56.199 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:56.211 INFO:teuthology.orchestra.run.smithi150.stdout:3328 write 0xe4bf210 thru 0xe4cacbc (0xbaad bytes) 2024-09-21T23:10:56.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:56.219 INFO:teuthology.orchestra.run.smithi150.stdout:3329 writesame 0x8239ec9 thru 0x82462fb data_size 0x1(0xc433 bytes) 2024-09-21T23:10:56.250 INFO:teuthology.orchestra.run.smithi150.stdout:3330 read 0x43f8423 thru 0x43ffdac (0x798a bytes) 2024-09-21T23:10:56.250 INFO:teuthology.orchestra.run.smithi150.stdout:3331 punch from 0x5468590 to 0x546fef5, (0x7965 bytes) 2024-09-21T23:10:56.250 INFO:teuthology.orchestra.run.smithi150.stdout:3332 read 0x850984d thru 0x850cd67 (0x351b bytes) 2024-09-21T23:10:56.250 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:56.248+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:56.250 INFO:teuthology.orchestra.run.smithi150.stdout:3334 read 0x7ae05a0 thru 0x7ae6624 (0x6085 bytes) 2024-09-21T23:10:56.252 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:56.252 INFO:teuthology.orchestra.run.smithi150.stdout:3337 punch from 0x3b12d65 to 0x3b1ed49, (0xbfe4 bytes) 2024-09-21T23:10:56.252 INFO:teuthology.orchestra.run.smithi150.stdout:3338 read 0xb0ac350 thru 0xb0b2aa6 (0x6757 bytes) 2024-09-21T23:10:56.253 INFO:teuthology.orchestra.run.smithi150.stdout:3340 punch from 0xac3c325 to 0xac48262, (0xbf3d bytes) 2024-09-21T23:10:56.253 INFO:teuthology.orchestra.run.smithi150.stdout:3341 punch from 0xc4f8a27 to 0xc506284, (0xd85d bytes) 2024-09-21T23:10:56.253 INFO:teuthology.orchestra.run.smithi150.stdout:3342 punch from 0x9187ce3 to 0x919034c, (0x8669 bytes) 2024-09-21T23:10:56.253 INFO:teuthology.orchestra.run.smithi150.stdout:3343 read 0xcebcfa8 thru 0xceca37e (0xd3d7 bytes) 2024-09-21T23:10:56.253 INFO:teuthology.orchestra.run.smithi150.stdout:3344 write 0x8b7b8ed thru 0x8b87392 (0xbaa6 bytes) 2024-09-21T23:10:56.263 INFO:teuthology.orchestra.run.smithi150.stdout:3345 read 0x286cef6 thru 0x28740ca (0x71d5 bytes) 2024-09-21T23:10:56.264 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:56.264 INFO:teuthology.orchestra.run.smithi150.stdout:3348 read 0x394b938 thru 0x3952566 (0x6c2f bytes) 2024-09-21T23:10:56.282 INFO:teuthology.orchestra.run.smithi150.stdout:3349 trunc from 0xe4cacbd to 0x408e5f7 2024-09-21T23:10:56.319 INFO:teuthology.orchestra.run.smithi150.stdout:3350 clone 28 order 24 su 1048576 sc 4 2024-09-21T23:10:56.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:10:56.359 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:56.359 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:56.359 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:56.356+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:56.374 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:10:56.460 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:56.460+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:56.535 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:56.532+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:56.539 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:56.539 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:56.553 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:10:56.640 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:10:56.640+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:10:56.716 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:56.716 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:56.731 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:10:56.736 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:56.735+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:56.837 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:56.835+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:56.898 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:56.898 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:56.913 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:10:56.960 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:56.960+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:57.037 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:57.037 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:57.038 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:57.035+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:57.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:10:57.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:10:57.139 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:57.135+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:57.211 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:57.211 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:57.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:10:57.272 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:57.272+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:57.276 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:57.277 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:57.293 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:57.294 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:57.311 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:57.311 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:57.329 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:57.329 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:57.330 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:10:57.334 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:57.341 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:57.339+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.278487105000004, 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:10:57.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:10:57.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.112720254000003, 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:57.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:10:57.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:10:57.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:38234 3.1e 3:7a5571f8:::rbd_data.1149ed0ffb75.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.384775099999999, 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043458748999999998, 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 38234 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968400+0000", 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6360000000000001e-06 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968406+0000", 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.81e-06 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968419+0000", 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3046e-05 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968426+0000", 2024-09-21T23:10:57.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5440000000000004e-06 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968519+0000", 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3348000000000001e-05 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968668+0000", 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148264 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.977244+0000", 2024-09-21T23:10:57.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085759600000000005 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011662+0000", 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034417917999999999 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011690+0000", 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8368999999999999e-05 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:10:57.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011780+0000", 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0450000000000003e-05 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011857+0000", 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6403999999999996e-05 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52271 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9948139149999999, 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039664149000000003, 2024-09-21T23:10:57.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52271 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358359+0000", 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358360+0000", 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358362+0000", 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9860000000000001e-06 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358368+0000", 2024-09-21T23:10:57.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358372+0000", 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7859999999999996e-06 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398016+0000", 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039643954000000002 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398023+0000", 2024-09-21T23:10:57.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9249999999999998e-06 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52394 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9945477389999997, 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039669796, 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52394 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358626+0000", 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6800000000000001e-07 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358628+0000", 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358634+0000", 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358639+0000", 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.772e-06 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398288+0000", 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039648640999999998 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398295+0000", 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0720000000000002e-06 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.378 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52395 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9945266069999992, 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039678049999999999, 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52395 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358647+0000", 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358649+0000", 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569999999999998e-06 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358655+0000", 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.127e-06 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358660+0000", 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6820000000000002e-06 2024-09-21T23:10:57.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398317+0000", 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039657215000000003 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398325+0000", 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4789999999999999e-06 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.382 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52396 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9945066699999998, 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039688093000000001, 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52396 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358667+0000", 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358669+0000", 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1509999999999998e-06 2024-09-21T23:10:57.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358676+0000", 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-06 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358681+0000", 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1420000000000001e-06 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398348+0000", 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039666202999999997 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398355+0000", 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.861e-06 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52397 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9944806869999994, 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039702474000000001, 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52397 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358693+0000", 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358695+0000", 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358701+0000", 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:10:57.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358706+0000", 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8550000000000001e-06 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398385+0000", 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039679157999999999 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398395+0000", 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9299999999999998e-06 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.389 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52398 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9944593719999997, 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039712074999999999, 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.390 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52398 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358716+0000", 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0940000000000002e-06 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358722+0000", 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-06 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358727+0000", 2024-09-21T23:10:57.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0760000000000002e-06 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398418+0000", 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039691048 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398426+0000", 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52399 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9944394610000007, 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039729353000000002, 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52399 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358736+0000", 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358743+0000", 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4119999999999998e-06 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358747+0000", 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8749999999999999e-06 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398453+0000", 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039705424000000003 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398463+0000", 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0182e-05 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52432 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9944194829999997, 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039740801999999999, 2024-09-21T23:10:57.397 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52432 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358756+0000", 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358762+0000", 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358767+0000", 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6129999999999997e-06 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398487+0000", 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039720278999999997 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398494+0000", 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5619999999999998e-06 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52433 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9943988820000005, 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039750774000000003, 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52433 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358775+0000", 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358777+0000", 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358783+0000", 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358788+0000", 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.135e-06 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398518+0000", 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039730389999999997 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398525+0000", 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52434 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9943791130000008, 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039771847999999999, 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52434 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358796+0000", 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8899999999999999e-06 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358802+0000", 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7139999999999998e-06 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358807+0000", 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8029999999999996e-06 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398556+0000", 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039749022000000002 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398566+0000", 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0067e-05 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52435 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9943594860000005, 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039820863999999997, 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52435 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8599999999999999e-07 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358816+0000", 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2180000000000002e-06 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358822+0000", 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9939999999999997e-06 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358827+0000", 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7670000000000003e-06 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398624+0000", 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797341 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398634+0000", 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0158e-05 2024-09-21T23:10:57.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52499 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9940708340000004, 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039762180000000001, 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52499 2024-09-21T23:10:57.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359103+0000", 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:10:57.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359105+0000", 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359111+0000", 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1120000000000001e-06 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359116+0000", 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398857+0000", 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039741132999999998 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398864+0000", 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5870000000000004e-06 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52500 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:57.416 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9940509760000005, 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776995000000002, 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52500 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:57.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359123+0000", 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359124+0000", 2024-09-21T23:10:57.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9750000000000001e-06 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359131+0000", 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359135+0000", 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7520000000000004e-06 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398892+0000", 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039756315 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398899+0000", 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.43e-06 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52526 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9940316980000006, 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039791576000000002, 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52526 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359142+0000", 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:10:57.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359144+0000", 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.838e-06 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359150+0000", 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359154+0000", 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398925+0000", 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039770497000000002 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398933+0000", 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1319999999999994e-06 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52527 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9940124560000001, 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03980852, 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52527 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359163+0000", 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.995e-06 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359169+0000", 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9710000000000001e-06 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359174+0000", 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0629999999999996e-06 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398961+0000", 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039786948000000003 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398969+0000", 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1580000000000005e-06 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.428 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52528 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9939928550000001, 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039822428999999999, 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52528 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359181+0000", 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359187+0000", 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359192+0000", 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7509999999999997e-06 2024-09-21T23:10:57.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359198+0000", 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9830000000000001e-06 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398995+0000", 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797036000000001 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399003+0000", 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:10:57.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52529 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9939677140000001, 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039831105999999998, 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:10:57.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52529 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359206+0000", 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359208+0000", 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359214+0000", 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1e-06 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359219+0000", 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7879999999999997e-06 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399029+0000", 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039810509000000001 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399036+0000", 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:10:57.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:10:57.437 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:57.437 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:57.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:10:57.439 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:57.439 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:57.439 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:57.440 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:57.440 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:57.473 INFO:teuthology.orchestra.run.smithi150.stdout:flattening image image_client.0-clone27 2024-09-21T23:10:57.474 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:57.472+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:57.542 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:57.539+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:57.580 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:57.580 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:57.580 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:57.580 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:57.595 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:10:57.687 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:57.687+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:57.722 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-21T23:10:57.744 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:57.744+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:57.759 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:57.759 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:57.760 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:57.772 INFO:teuthology.orchestra.run.smithi150.stdout:3351 write 0xd42fac5 thru 0xd43353e (0x3a7a bytes) 2024-09-21T23:10:57.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:10:57.778 INFO:teuthology.orchestra.run.smithi150.stdout:3352 punch from 0x1dfd86d to 0x1e036dc, (0x5e6f bytes) 2024-09-21T23:10:57.779 INFO:teuthology.orchestra.run.smithi150.stdout:3353 write 0x9e47111 thru 0x9e4ede6 (0x7cd6 bytes) 2024-09-21T23:10:57.783 INFO:teuthology.orchestra.run.smithi150.stdout:3354 write 0x3e53ef7 thru 0x3e56516 (0x2620 bytes) 2024-09-21T23:10:57.808 INFO:teuthology.orchestra.run.smithi150.stdout:3355 trunc from 0xd43353f to 0x9da1353 2024-09-21T23:10:57.828 INFO:teuthology.orchestra.run.smithi150.stdout:3356 writesame 0x47ffc81 thru 0x4809107 data_size 0x1(0x9487 bytes) 2024-09-21T23:10:57.845 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:57.844+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:57.893 INFO:teuthology.orchestra.run.smithi150.stdout:3357 trunc from 0x9da1353 to 0x3f1ce2b 2024-09-21T23:10:57.930 INFO:teuthology.orchestra.run.smithi150.stdout:3358 punch from 0x3afed66 to 0x3b02ec9, (0x4163 bytes) 2024-09-21T23:10:57.936 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:57.936 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:57.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:57.936 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:57.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.936 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5124 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.749394821999999, 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046086192999999998, 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5124 2024-09-21T23:10:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186880+0000", 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186881+0000", 2024-09-21T23:10:57.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.133e-06 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186887+0000", 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7939999999999999e-06 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186904+0000", 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7193e-05 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.186918+0000", 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T23:10:57.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223027+0000", 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036108460000000002 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.223201+0000", 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017483900000000001 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.232966+0000", 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097649039999999996 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.940 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5279 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.748116181999997, 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045406334999999999, 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5279 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.941 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188159+0000", 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188160+0000", 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T23:10:57.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188167+0000", 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0700000000000001e-06 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188188+0000", 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-05 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188205+0000", 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7444000000000001e-05 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233051+0000", 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044846496999999999 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233167+0000", 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115307 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233565+0000", 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039829399999999999 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5280 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.748046985999999, 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047857909999999997, 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5280 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188228+0000", 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188229+0000", 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.237e-06 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188235+0000", 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188254+0000", 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9015999999999999e-05 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188268+0000", 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.395e-05 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233608+0000", 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045339957 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.233722+0000", 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011403700000000001 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236086+0000", 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00236359 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5281 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.747985485999997, 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048399091999999998, 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5281 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188289+0000", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188291+0000", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188298+0000", 2024-09-21T23:10:57.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188315+0000", 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7819999999999999e-05 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188329+0000", 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3383e-05 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236130+0000", 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047800827999999997 2024-09-21T23:10:57.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236240+0000", 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110745 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236689+0000", 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044823600000000002 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5282 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.952 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.747927607000001, 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048602487, 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5282 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188347+0000", 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188348+0000", 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.144e-06 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188355+0000", 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3029999999999996e-06 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188373+0000", 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8056000000000001e-05 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188386+0000", 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3536999999999999e-05 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236748+0000", 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048361617000000003 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236840+0000", 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2075000000000002e-05 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236950+0000", 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109755 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5283 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.747784348000003, 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048844519000000003, 2024-09-21T23:10:57.956 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5283 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188491+0000", 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188493+0000", 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3800000000000001e-06 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188503+0000", 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188526+0000", 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.304e-05 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188543+0000", 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7557000000000001e-05 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237020+0000", 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048476656 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237187+0000", 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016670199999999999 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237335+0000", 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014857100000000001 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5284 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:57.960 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.747702482000001, 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049117881000000002, 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5284 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188572+0000", 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188574+0000", 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188580+0000", 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188598+0000", 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8304e-05 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188611+0000", 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3329000000000001e-05 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237402+0000", 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048790838000000003 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237549+0000", 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146897 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237690+0000", 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141383 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.964 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5285 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.747645196000001, 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049392706000000001, 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5285 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:57.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188630+0000", 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188631+0000", 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0400000000000005e-07 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188636+0000", 2024-09-21T23:10:57.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8329999999999999e-06 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188655+0000", 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8246000000000001e-05 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188670+0000", 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5440000000000001e-05 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237748+0000", 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049077975000000003 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.237890+0000", 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141953 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238022+0000", 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013235499999999999 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5286 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:57.968 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.747586093000002, 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049655168, 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5286 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188689+0000", 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188690+0000", 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1769999999999999e-06 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188696+0000", 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5269999999999998e-06 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188713+0000", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7836e-05 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.188726+0000", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238077+0000", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049350964999999997 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238228+0000", 2024-09-21T23:10:57.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015027000000000001 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238344+0000", 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116403 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5287 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.746816504999998, 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049431115999999997, 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5287 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189458+0000", 2024-09-21T23:10:57.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189460+0000", 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189470+0000", 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189491+0000", 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0743000000000001e-05 2024-09-21T23:10:57.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.189503+0000", 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2787e-05 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238395+0000", 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048891284 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238519+0000", 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124745 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238890+0000", 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037020400000000001 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5465 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2916352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.739020513, 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045516793999999999, 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5465 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197254+0000", 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197256+0000", 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197262+0000", 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0340000000000002e-06 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197280+0000", 2024-09-21T23:10:57.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372999999999998e-05 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197294+0000", 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3983e-05 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242417+0000", 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045122664999999999 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242550+0000", 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013329499999999999 2024-09-21T23:10:57.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242771+0000", 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022120399999999999 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5466 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3375104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.738962456000003, 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045852049999999998, 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.980 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5466 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197312+0000", 2024-09-21T23:10:57.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197313+0000", 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197321+0000", 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197338+0000", 2024-09-21T23:10:57.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7824000000000001e-05 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197353+0000", 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.456e-05 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.242853+0000", 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045500119999999998 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243010+0000", 2024-09-21T23:10:57.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015682599999999999 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243165+0000", 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015463400000000001 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5467 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.738903213, 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046123232, 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.984 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5467 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197372+0000", 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197373+0000", 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197378+0000", 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197397+0000", 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8793000000000001e-05 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.197411+0000", 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3726999999999999e-05 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243249+0000", 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045838320000000002 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243376+0000", 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012760199999999999 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.243495+0000", 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011848100000000001 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5639 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:57.988 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.736091602000002, 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045343482999999997, 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5639 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200183+0000", 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200184+0000", 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200190+0000", 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:10:57.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200209+0000", 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8563000000000002e-05 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200223+0000", 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4173e-05 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.244967+0000", 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044743567999999997 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245103+0000", 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013634300000000001 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245527+0000", 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042370199999999998 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5640 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.992 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.735995744999997, 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045524901, 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5640 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:57.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200279+0000", 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200280+0000", 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.24e-06 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200287+0000", 2024-09-21T23:10:57.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200307+0000", 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0129999999999999e-05 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200321+0000", 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.384e-05 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245576+0000", 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045255521 2024-09-21T23:10:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245701+0000", 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012480100000000001 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245804+0000", 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102933 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5641 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:57.996 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.735933105999997, 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045750444000000001, 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5641 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200342+0000", 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200343+0000", 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200351+0000", 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9160000000000001e-06 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200369+0000", 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8490000000000001e-05 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200383+0000", 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3234000000000001e-05 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245850+0000", 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045467883000000001 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:57.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.245979+0000", 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128641 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246092+0000", 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113143 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5642 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.735873003000002, 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045975347999999999, 2024-09-21T23:10:58.000 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5642 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200402+0000", 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200403+0000", 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200409+0000", 2024-09-21T23:10:58.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6570000000000002e-06 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200427+0000", 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8306000000000001e-05 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200441+0000", 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4042999999999999e-05 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246144+0000", 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.045702495000000003 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246277+0000", 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133831 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246377+0000", 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9835999999999998e-05 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5643 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 2949120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.735815477000003, 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046409156, 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5643 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200459+0000", 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200460+0000", 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200466+0000", 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200483+0000", 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7776999999999999e-05 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200500+0000", 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6852999999999999e-05 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246582+0000", 2024-09-21T23:10:58.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046081471999999998 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246725+0000", 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143184 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246869+0000", 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014361699999999999 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5644 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:58.008 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.735755404000002, 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046601900000000002, 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5644 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200520+0000", 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200521+0000", 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.407e-06 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200526+0000", 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3499999999999996e-06 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200544+0000", 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7410000000000001e-05 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200559+0000", 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5435999999999999e-05 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.246920+0000", 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046360902000000002 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247055+0000", 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468200000000001 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247121+0000", 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6712999999999996e-05 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5645 3.c 3:3260709e:::rbd_data.114912f5d0f.000000000000000a:7 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.735696677999996, 2024-09-21T23:10:58.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.04674565, 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5645 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200578+0000", 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200579+0000", 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2270000000000001e-06 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200585+0000", 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8300000000000001e-06 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200605+0000", 2024-09-21T23:10:58.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9675999999999999e-05 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.200618+0000", 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509999999999999e-05 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247148+0000", 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046529451999999999 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247219+0000", 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0833000000000004e-05 2024-09-21T23:10:58.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.247324+0000", 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105122 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.016 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:58.017 INFO:teuthology.orchestra.run.smithi150.stdout:3359 writesame 0xd8f125b thru 0xd8f87a9 data_size 0x1(0x754f bytes) 2024-09-21T23:10:58.018 INFO:teuthology.orchestra.run.smithi150.stdout:3361 write 0xe9372de thru 0xe9469c7 (0xf6ea bytes) 2024-09-21T23:10:58.018 INFO:teuthology.orchestra.run.smithi150.stdout:3363 trunc from 0xe9469c8 to 0xd5487b4 2024-09-21T23:10:58.018 INFO:teuthology.orchestra.run.smithi150.stdout:3365 read 0x58793e5 thru 0x587ab33 (0x174f bytes) 2024-09-21T23:10:58.018 INFO:teuthology.orchestra.run.smithi150.stdout:3366 write 0xae250e7 thru 0xae26c37 (0x1b51 bytes) 2024-09-21T23:10:58.018 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:58.016+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:58.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:10:58.021 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.021 INFO:teuthology.orchestra.run.smithi150.stdout:3369 read 0x3f5b22a thru 0x3f5e15e (0x2f35 bytes) 2024-09-21T23:10:58.032 INFO:teuthology.orchestra.run.smithi150.stdout:3371 trunc from 0xd5487b4 to 0x9d2a1a6 2024-09-21T23:10:58.078 INFO:teuthology.orchestra.run.smithi150.stdout:3372 write 0xba3374b thru 0xba3688c (0x3142 bytes) 2024-09-21T23:10:58.081 INFO:teuthology.orchestra.run.smithi150.stdout:3373 writesame 0x220fa1a thru 0x2213999 data_size 0x20(0x3f80 bytes) 2024-09-21T23:10:58.085 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.085 INFO:teuthology.orchestra.run.smithi150.stdout:3378 write 0x3a95a93 thru 0x3aa458c (0xeafa bytes) 2024-09-21T23:10:58.087 INFO:teuthology.orchestra.run.smithi150.stdout:3379 write 0x652732a thru 0x653455c (0xd233 bytes) 2024-09-21T23:10:58.094 INFO:teuthology.orchestra.run.smithi150.stdout:3380 read 0x1fc3551 thru 0x1fccfa4 (0x9a54 bytes) 2024-09-21T23:10:58.096 INFO:teuthology.orchestra.run.smithi150.stdout:3381 write 0x51d6c44 thru 0x51e59fb (0xedb8 bytes) 2024-09-21T23:10:58.099 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.099 INFO:teuthology.orchestra.run.smithi150.stdout:3383 punch from 0x3c6061 to 0x3d0efb, (0xae9a bytes) 2024-09-21T23:10:58.099 INFO:teuthology.orchestra.run.smithi150.stdout:3384 read 0x6d08b99 thru 0x6d15c1b (0xd083 bytes) 2024-09-21T23:10:58.100 INFO:teuthology.orchestra.run.smithi150.stdout:3387 read 0x175d82a thru 0x175ee57 (0x162e bytes) 2024-09-21T23:10:58.101 INFO:teuthology.orchestra.run.smithi150.stdout:3388 write 0x3e87db4 thru 0x3e8f016 (0x7263 bytes) 2024-09-21T23:10:58.105 INFO:teuthology.orchestra.run.smithi150.stdout:3389 read 0x59b0c28 thru 0x59b6dc0 (0x6199 bytes) 2024-09-21T23:10:58.106 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.106 INFO:teuthology.orchestra.run.smithi150.stdout:3391 punch from 0x5b0e764 to 0x5b1c76d, (0xe009 bytes) 2024-09-21T23:10:58.106 INFO:teuthology.orchestra.run.smithi150.stdout:3392 punch from 0x936f0e0 to 0x937b112, (0xc032 bytes) 2024-09-21T23:10:58.106 INFO:teuthology.orchestra.run.smithi150.stdout:3395 read 0x37898fd thru 0x378a4d7 (0xbdb bytes) 2024-09-21T23:10:58.118 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:58.115+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:58.120 INFO:teuthology.orchestra.run.smithi150.stdout:3398 trunc from 0xba3688d to 0x6d08a20 2024-09-21T23:10:58.147 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:58.147 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:58.147 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:58.147 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:58.148 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:10:58.167 INFO:teuthology.orchestra.run.smithi150.stdout:3400 trunc from 0x6d08a20 to 0xdbc6603 2024-09-21T23:10:58.184 INFO:teuthology.orchestra.run.smithi150.stdout:3401 punch from 0x50aedac to 0x50b1523, (0x2777 bytes) 2024-09-21T23:10:58.184 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.184 INFO:teuthology.orchestra.run.smithi150.stdout:3405 punch from 0xd4188ae to 0xd41a3ce, (0x1b20 bytes) 2024-09-21T23:10:58.184 INFO:teuthology.orchestra.run.smithi150.stdout:3406 write 0x636dd7a thru 0x637c835 (0xeabc bytes) 2024-09-21T23:10:58.191 INFO:teuthology.orchestra.run.smithi150.stdout:3409 punch from 0x2ab25f9 to 0x2abd527, (0xaf2e bytes) 2024-09-21T23:10:58.191 INFO:teuthology.orchestra.run.smithi150.stdout:3412 write 0x5f22ddd thru 0x5f2fc27 (0xce4b bytes) 2024-09-21T23:10:58.210 INFO:teuthology.orchestra.run.smithi150.stdout:3413 write 0xde935b1 thru 0xde9aca7 (0x76f7 bytes) 2024-09-21T23:10:58.215 INFO:teuthology.orchestra.run.smithi150.stdout:3414 write 0xdbe22cf thru 0xdbe6efa (0x4c2c bytes) 2024-09-21T23:10:58.219 INFO:teuthology.orchestra.run.smithi150.stdout:3416 punch from 0x1b73537 to 0x1b7875f, (0x5228 bytes) 2024-09-21T23:10:58.220 INFO:teuthology.orchestra.run.smithi150.stdout:3417 write 0x2ad537a thru 0x2ad614e (0xdd5 bytes) 2024-09-21T23:10:58.223 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.225 INFO:teuthology.orchestra.run.smithi150.stdout:3420 trunc from 0xde9aca8 to 0xd359d88 2024-09-21T23:10:58.242 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.242 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.242 INFO:teuthology.orchestra.run.smithi150.stdout:3423 write 0xcb5c817 thru 0xcb654cd (0x8cb7 bytes) 2024-09-21T23:10:58.244 INFO:teuthology.orchestra.run.smithi150.stdout:3424 writesame 0x241fd9f thru 0x24280b5 data_size 0x1(0x8317 bytes) 2024-09-21T23:10:58.247 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:58.247+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:58.274 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.274 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.274 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.274 INFO:teuthology.orchestra.run.smithi150.stdout:3429 writesame 0x3db1b27 thru 0x3db86f3 data_size 0x1(0x6bcd bytes) 2024-09-21T23:10:58.303 INFO:teuthology.orchestra.run.smithi150.stdout:3430 write 0xc28aead thru 0xc293e80 (0x8fd4 bytes) 2024-09-21T23:10:58.306 INFO:teuthology.orchestra.run.smithi150.stdout:3431 punch from 0x89f5ce5 to 0x89f977c, (0x3a97 bytes) 2024-09-21T23:10:58.306 INFO:teuthology.orchestra.run.smithi150.stdout:3434 write 0x30fd7d0 thru 0x310208f (0x48c0 bytes) 2024-09-21T23:10:58.309 INFO:teuthology.orchestra.run.smithi150.stdout:3438 read 0xb6193fe thru 0xb61b8a3 (0x24a6 bytes) 2024-09-21T23:10:58.309 INFO:teuthology.orchestra.run.smithi150.stdout:3439 read 0x3e8945b thru 0x3e91885 (0x842b bytes) 2024-09-21T23:10:58.311 INFO:teuthology.orchestra.run.smithi150.stdout:3441 write 0x5d46e5d thru 0x5d55d58 (0xeefc bytes) 2024-09-21T23:10:58.314 INFO:teuthology.orchestra.run.smithi150.stdout:3442 read 0x8fbb557 thru 0x8fc14f8 (0x5fa2 bytes) 2024-09-21T23:10:58.314 INFO:teuthology.orchestra.run.smithi150.stdout:3443 read 0xa921298 thru 0xa92602c (0x4d95 bytes) 2024-09-21T23:10:58.314 INFO:teuthology.orchestra.run.smithi150.stdout:3444 punch from 0x764fd10 to 0x765ef2a, (0xf21a bytes) 2024-09-21T23:10:58.314 INFO:teuthology.orchestra.run.smithi150.stdout:3445 read 0x6d9a3af thru 0x6da1fff (0x7c51 bytes) 2024-09-21T23:10:58.315 INFO:teuthology.orchestra.run.smithi150.stdout:3447 write 0xcfe0a3c thru 0xcfe4358 (0x391d bytes) 2024-09-21T23:10:58.317 INFO:teuthology.orchestra.run.smithi150.stdout:3448 write 0x9e72978 thru 0x9e73aae (0x1137 bytes) 2024-09-21T23:10:58.321 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:58.320+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:58.323 INFO:teuthology.orchestra.run.smithi150.stdout:3449 read 0xad6869b thru 0xad74673 (0xbfd9 bytes) 2024-09-21T23:10:58.336 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:58.337 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:58.337 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:58.337 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:58.337 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:58.340 INFO:teuthology.orchestra.run.smithi150.stdout:3451 write 0xe96c9ff thru 0xe96e47b (0x1a7d bytes) 2024-09-21T23:10:58.345 INFO:teuthology.orchestra.run.smithi150.stdout:3453 write 0x34b9d3f thru 0x34c9185 (0xf447 bytes) 2024-09-21T23:10:58.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:10:58.378 INFO:teuthology.orchestra.run.smithi150.stdout:3454 trunc from 0xe96e47c to 0x372b1af 2024-09-21T23:10:58.400 INFO:teuthology.orchestra.run.smithi150.stdout:3455 punch from 0xb650c0 to 0xb65600, (0x540 bytes) 2024-09-21T23:10:58.400 INFO:teuthology.orchestra.run.smithi150.stdout:3456 punch from 0x329f576 to 0x32a726d, (0x7cf7 bytes) 2024-09-21T23:10:58.400 INFO:teuthology.orchestra.run.smithi150.stdout:3458 read 0x21ee37a thru 0x21f48b3 (0x653a bytes) 2024-09-21T23:10:58.401 INFO:teuthology.orchestra.run.smithi150.stdout:3459 write 0xe3b1b thru 0xf02b0 (0xc796 bytes) 2024-09-21T23:10:58.413 INFO:teuthology.orchestra.run.smithi150.stdout:3460 trunc from 0x372b1af to 0x73acc14 2024-09-21T23:10:58.439 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:58.439+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:58.453 INFO:teuthology.orchestra.run.smithi150.stdout:3461 write 0xca7b8e8 thru 0xca7df4a (0x2663 bytes) 2024-09-21T23:10:58.490 INFO:teuthology.orchestra.run.smithi150.stdout:3462 trunc from 0xca7df4b to 0x275096 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:27644 3.10 3:0811e7db:::rbd_data.11499b8b109.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 37.164937969999997, 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041757458999999997, 2024-09-21T23:10:58.519 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 27644 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353814+0000", 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.353815+0000", 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.426e-06 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359473+0000", 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056580709999999998 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359492+0000", 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8978999999999999e-05 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359500+0000", 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0909999999999993e-06 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359556+0000", 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6317999999999998e-05 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.359660+0000", 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103915 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0", 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.360026+0000", 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00036547400000000002 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.370944+0000", 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010918323000000001 2024-09-21T23:10:58.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395472+0000", 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024528372999999999 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395527+0000", 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4873999999999997e-05 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:21.395571+0000", 2024-09-21T23:10:58.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3615000000000002e-05 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.791448792000001, 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.525 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:10:58.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:10:58.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39778 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8396800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.500689+0000", 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.018062194000001, 2024-09-21T23:10:58.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030452587, 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39778 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500689+0000", 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500689+0000", 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500690+0000", 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1960000000000001e-06 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500696+0000", 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6969999999999998e-06 2024-09-21T23:10:58.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500715+0000", 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8372e-05 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500729+0000", 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3959e-05 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.530824+0000", 2024-09-21T23:10:58.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030095093 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.530940+0000", 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115989 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531142+0000", 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020228100000000001 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39779 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8511488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.500749+0000", 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.018002985000001, 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030774979000000001, 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39779 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500749+0000", 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500749+0000", 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500750+0000", 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.164e-06 2024-09-21T23:10:58.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500755+0000", 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.293e-06 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500772+0000", 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7452000000000002e-05 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500786+0000", 2024-09-21T23:10:58.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3232e-05 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531215+0000", 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030429540000000001 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531326+0000", 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110703 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531523+0000", 2024-09-21T23:10:58.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019759500000000001 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39780 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8454144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.500820+0000", 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017931056, 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031111728000000002, 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.540 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39780 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500820+0000", 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500820+0000", 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500822+0000", 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1570000000000001e-06 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500840+0000", 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8742000000000001e-05 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500859+0000", 2024-09-21T23:10:58.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8989000000000001e-05 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.500874+0000", 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4226000000000001e-05 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531594+0000", 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030720254999999998 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531715+0000", 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012127499999999999 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.531932+0000", 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021708400000000001 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39791 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8544256~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017681934000001, 2024-09-21T23:10:58.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031313771999999997, 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39791 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501070+0000", 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7000000000000003e-07 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501074+0000", 2024-09-21T23:10:58.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501087+0000", 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3115999999999999e-05 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501094+0000", 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4760000000000001e-06 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532017+0000", 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030923302999999999 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532167+0000", 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000149424 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532383+0000", 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021645 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.548 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39792 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8601600~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017646922999999, 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031745609000000001, 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39792 2024-09-21T23:10:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501105+0000", 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9400000000000004e-07 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501109+0000", 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501118+0000", 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7589999999999992e-06 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501126+0000", 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0630000000000006e-06 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532468+0000", 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031341130000000002 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532605+0000", 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013782200000000001 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532850+0000", 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024479800000000001 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39800 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8011776~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501231+0000", 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017520241, 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032088402000000002, 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.553 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39800 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501231+0000", 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501231+0000", 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501232+0000", 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501235+0000", 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0860000000000002e-06 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501245+0000", 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7059999999999999e-06 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501252+0000", 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9e-06 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.532953+0000", 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031701288000000001 2024-09-21T23:10:58.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533105+0000", 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000151705 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533320+0000", 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021504299999999999 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39801 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8069120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017486793, 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032634899000000002, 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39801 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501265+0000", 2024-09-21T23:10:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501268+0000", 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9670000000000001e-06 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501281+0000", 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2704999999999999e-05 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501288+0000", 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.114e-06 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533394+0000", 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032106226000000002 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533510+0000", 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0001159 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.533900+0000", 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038933000000000001 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39814 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8159232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017379736000001, 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033410207999999997, 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39814 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501372+0000", 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1999999999999999e-07 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501376+0000", 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.146e-06 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501385+0000", 2024-09-21T23:10:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7100000000000002e-06 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501393+0000", 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2030000000000003e-06 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534062+0000", 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032669863 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534352+0000", 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028966199999999999 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534782+0000", 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000429904 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39815 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8216576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501402+0000", 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017349004, 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033993651999999999, 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39815 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501402+0000", 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501402+0000", 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501403+0000", 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501407+0000", 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.189e-06 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501424+0000", 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6422999999999998e-05 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501450+0000", 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5962e-05 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.534913+0000", 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033463360999999997 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535138+0000", 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000224849 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535396+0000", 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025820800000000001 2024-09-21T23:10:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39828 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8306688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501642+0000", 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017109086, 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034518844, 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39828 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501642+0000", 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501642+0000", 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501644+0000", 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1999999999999999e-06 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501649+0000", 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.524e-06 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501667+0000", 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7682000000000001e-05 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501680+0000", 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2962000000000001e-05 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535508+0000", 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033828035999999999 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.535723+0000", 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000215267 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536161+0000", 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043817299999999998 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:39829 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 8364032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:43.501698+0000", 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.017053348999999, 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034896215000000001, 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39829 2024-09-21T23:10:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501698+0000", 2024-09-21T23:10:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501698+0000", 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501699+0000", 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.054e-06 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501705+0000", 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6550000000000001e-06 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501722+0000", 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7393000000000001e-05 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.501735+0000", 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3189e-05 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536223+0000", 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034487732 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536365+0000", 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142282 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:43.536594+0000", 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022891 2024-09-21T23:10:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41982 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 3956736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.422112842000001, 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030321332999999999, 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41982 2024-09-21T23:10:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096639+0000", 2024-09-21T23:10:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096644+0000", 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4669999999999998e-06 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096661+0000", 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6558999999999998e-05 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096671+0000", 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0689e-05 2024-09-21T23:10:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.126521+0000", 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.029849496 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.126681+0000", 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000160438 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.126960+0000", 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027883399999999999 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41983 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4014080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096695+0000", 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.422056012000001, 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030781096000000001, 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41983 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096695+0000", 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096695+0000", 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096696+0000", 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2600000000000001e-07 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096701+0000", 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5859999999999998e-06 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096716+0000", 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4901e-05 2024-09-21T23:10:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096727+0000", 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1551e-05 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127021+0000", 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030293433000000002 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127181+0000", 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016025100000000001 2024-09-21T23:10:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127477+0000", 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029544800000000002 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41996 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4104192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096862+0000", 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.421889315, 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031125463999999999, 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41996 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096862+0000", 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096862+0000", 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096863+0000", 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096868+0000", 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1760000000000001e-06 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096883+0000", 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4938e-05 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096894+0000", 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0329999999999999e-05 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127539+0000", 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030645596000000001 2024-09-21T23:10:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127701+0000", 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016159099999999999 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.127988+0000", 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028668399999999999 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:41997 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4161536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.096911+0000", 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.421840043, 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.03158888, 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 41997 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096911+0000", 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096911+0000", 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096912+0000", 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096917+0000", 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0540000000000002e-06 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096934+0000", 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6070999999999999e-05 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.096957+0000", 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3924e-05 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128048+0000", 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031090824999999999 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128212+0000", 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016409199999999999 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128500+0000", 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028793900000000002 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:42009 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4194304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.097094+0000", 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.421657893999999, 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031926768000000001, 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 42009 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097094+0000", 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097094+0000", 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097095+0000", 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.17e-06 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097100+0000", 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.485e-06 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097116+0000", 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5559000000000001e-05 2024-09-21T23:10:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097128+0000", 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1786e-05 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128563+0000", 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031434974999999997 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.128722+0000", 2024-09-21T23:10:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015934099999999999 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129020+0000", 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029845199999999999 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:42010 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4251648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.097146+0000", 2024-09-21T23:10:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.42160573, 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032297840000000001, 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 42010 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097146+0000", 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097146+0000", 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097147+0000", 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3099999999999996e-07 2024-09-21T23:10:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097163+0000", 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5835999999999999e-05 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097177+0000", 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4681999999999999e-05 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097185+0000", 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8140000000000001e-06 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129106+0000", 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031920945999999999 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129228+0000", 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012197399999999999 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129444+0000", 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021565700000000001 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:42012 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:18 [read 4308992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:44.097223+0000", 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 14.421528670000001, 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032712365, 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 42012 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097223+0000", 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097223+0000", 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097224+0000", 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2299999999999999e-07 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097228+0000", 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.527e-06 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097240+0000", 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1605e-05 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.097247+0000", 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129498+0000", 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032251552000000003 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129654+0000", 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015591499999999999 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:44.129935+0000", 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028121100000000002 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:58.611 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:58.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:10:58.614 INFO:teuthology.orchestra.run.smithi150.stdout:3463 write 0xa7ac9f1 thru 0xa7b4228 (0x7838 bytes) 2024-09-21T23:10:58.614 INFO:teuthology.orchestra.run.smithi150.stdout:3464 trunc from 0xa7b4229 to 0xdd82447 2024-09-21T23:10:58.614 INFO:teuthology.orchestra.run.smithi150.stdout:3465 write 0x6c2d182 thru 0x6c3b309 (0xe188 bytes) 2024-09-21T23:10:58.614 INFO:teuthology.orchestra.run.smithi150.stdout:3467 read 0x8abaadb thru 0x8ac7269 (0xc78f bytes) 2024-09-21T23:10:58.614 INFO:teuthology.orchestra.run.smithi150.stdout:3468 writesame 0xd9cc3a3 thru 0xd9cc953 data_size 0x1(0x5b1 bytes) 2024-09-21T23:10:58.614 INFO:teuthology.orchestra.run.smithi150.stdout:3469 writesame 0x8f34a2e thru 0x8f436f5 data_size 0x4(0xecc8 bytes) 2024-09-21T23:10:58.614 INFO:teuthology.orchestra.run.smithi150.stdout:3472 write 0x7d41fe3 thru 0x7d49c4e (0x7c6c bytes) 2024-09-21T23:10:58.615 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:58.612+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:58.619 INFO:teuthology.orchestra.run.smithi150.stdout:3474 write 0xe4d5bfc thru 0xe4e561f (0xfa24 bytes) 2024-09-21T23:10:58.625 INFO:teuthology.orchestra.run.smithi150.stdout:3475 read 0x13de3e9 thru 0x13e6164 (0x7d7c bytes) 2024-09-21T23:10:58.654 INFO:teuthology.orchestra.run.smithi150.stdout:3476 trunc from 0xe4e5620 to 0x383c3dd 2024-09-21T23:10:58.683 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.683 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.683 INFO:teuthology.orchestra.run.smithi150.stdout:3481 read 0x1fc429d thru 0x1fc8982 (0x46e6 bytes) 2024-09-21T23:10:58.714 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:10:58.712+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:10:58.719 INFO:teuthology.orchestra.run.smithi150.stdout:3482 writesame 0xc743949 thru 0xc75015d data_size 0x17(0xc815 bytes) 2024-09-21T23:10:58.719 INFO:teuthology.orchestra.run.smithi150.stdout:3483 punch from 0xb00ac0e to 0xb00b7cc, (0xbbe bytes) 2024-09-21T23:10:58.719 INFO:teuthology.orchestra.run.smithi150.stdout:3484 read 0x481ec82 thru 0x482d119 (0xe498 bytes) 2024-09-21T23:10:58.719 INFO:teuthology.orchestra.run.smithi150.stdout:3485 punch from 0x1310717 to 0x131f68d, (0xef76 bytes) 2024-09-21T23:10:58.719 INFO:teuthology.orchestra.run.smithi150.stdout:3486 punch from 0x50755cf to 0x50813b4, (0xbde5 bytes) 2024-09-21T23:10:58.719 INFO:teuthology.orchestra.run.smithi150.stdout:3487 read 0x8c6023f thru 0x8c669ad (0x676f bytes) 2024-09-21T23:10:58.743 INFO:teuthology.orchestra.run.smithi150.stdout:3488 trunc from 0xc75015e to 0x227b2bb 2024-09-21T23:10:58.759 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.759 INFO:teuthology.orchestra.run.smithi150.stdout:3491 writesame 0xd03fb9 thru 0xd08663 data_size 0x1(0x46ab bytes) 2024-09-21T23:10:58.784 INFO:teuthology.orchestra.run.smithi150.stdout:3492 read 0x197456f thru 0x19766c2 (0x2154 bytes) 2024-09-21T23:10:58.786 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:58.786 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:58.786 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:10:58.787 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:58.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:10:58.813 INFO:teuthology.orchestra.run.smithi150.stdout:3493 write 0x7319d4d thru 0x732224c (0x8500 bytes) 2024-09-21T23:10:58.814 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:58.812+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:58.834 INFO:teuthology.orchestra.run.smithi150.stdout:3494 writesame 0x80da392 thru 0x80dcbf4 data_size 0x7(0x2863 bytes) 2024-09-21T23:10:58.834 INFO:teuthology.orchestra.run.smithi150.stdout:3495 read 0xd8bac8 thru 0xd8c862 (0xd9b bytes) 2024-09-21T23:10:58.834 INFO:teuthology.orchestra.run.smithi150.stdout:3497 punch from 0x1fbcc76 to 0x1fbef2b, (0x22b5 bytes) 2024-09-21T23:10:58.835 INFO:teuthology.orchestra.run.smithi150.stdout:3498 read 0x4fe5c51 thru 0x4fed8ac (0x7c5c bytes) 2024-09-21T23:10:58.835 INFO:teuthology.orchestra.run.smithi150.stdout:3499 read 0x3f0b3cb thru 0x3f0e8b4 (0x34ea bytes) 2024-09-21T23:10:58.835 INFO:teuthology.orchestra.run.smithi150.stdout:3500 read 0x12391b2 thru 0x1239257 (0xa6 bytes) 2024-09-21T23:10:58.837 INFO:teuthology.orchestra.run.smithi150.stdout:3501 read 0x7186079 thru 0x719412b (0xe0b3 bytes) 2024-09-21T23:10:58.860 INFO:teuthology.orchestra.run.smithi150.stdout:3502 write 0xbff4b27 thru 0xbffa237 (0x5711 bytes) 2024-09-21T23:10:58.874 INFO:teuthology.orchestra.run.smithi150.stdout:3505 trunc from 0xbffa238 to 0x8c482da 2024-09-21T23:10:58.894 INFO:teuthology.orchestra.run.smithi150.stdout:3507 trunc from 0x8c482da to 0xa73e5a7 2024-09-21T23:10:58.906 INFO:teuthology.orchestra.run.smithi150.stdout:3511 punch from 0x648a302 to 0x648ea69, (0x4767 bytes) 2024-09-21T23:10:58.906 INFO:teuthology.orchestra.run.smithi150.stdout:3512 writesame 0x6c124b thru 0x6c145b data_size 0x1(0x211 bytes) 2024-09-21T23:10:58.908 INFO:teuthology.orchestra.run.smithi150.stdout:3513 read 0x54683b0 thru 0x5468580 (0x1d1 bytes) 2024-09-21T23:10:58.908 INFO:teuthology.orchestra.run.smithi150.stdout:3516 punch from 0x9c9ba3f to 0x9ca8002, (0xc5c3 bytes) 2024-09-21T23:10:58.915 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:10:58.912+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:10:58.922 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:58.922 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:10:58.922 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:10:58.922 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:10:58.923 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:58.935 INFO:teuthology.orchestra.run.smithi150.stdout:3517 trunc from 0xa73e5a7 to 0x9cc884 2024-09-21T23:10:58.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:10:58.990 INFO:teuthology.orchestra.run.smithi150.stdout:3518 write 0x7285107 thru 0x728daaa (0x89a4 bytes) 2024-09-21T23:10:58.991 INFO:teuthology.orchestra.run.smithi150.stdout:3519 punch from 0x1d9de4b to 0x1da6a24, (0x8bd9 bytes) 2024-09-21T23:10:58.991 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:58.994 INFO:teuthology.orchestra.run.smithi150.stdout:3521 trunc from 0x728daab to 0x8421b6d 2024-09-21T23:10:59.015 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:59.015+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:59.023 INFO:teuthology.orchestra.run.smithi150.stdout:3522 write 0xa211a8a thru 0xa215457 (0x39ce bytes) 2024-09-21T23:10:59.030 INFO:teuthology.orchestra.run.smithi150.stdout:3523 read 0xea0bfd thru 0xea23ab (0x17af bytes) 2024-09-21T23:10:59.031 INFO:teuthology.orchestra.run.smithi150.stdout:3525 read 0x809c6f5 thru 0x80a4140 (0x7a4c bytes) 2024-09-21T23:10:59.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:10:59.050 INFO:teuthology.orchestra.run.smithi150.stdout:3526 trunc from 0xa215458 to 0x3396e74 2024-09-21T23:10:59.060 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:10:59.060 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:10:59.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.875889213000001, 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.061 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:10:59.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:10:59.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.875786126999998, 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.065 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:10:59.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:10:59.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:10:59.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.875718778, 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.800531841999998, 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:10:59.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:10:59.076 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.800476074000002, 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:10:59.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:10:59.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:10:59.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.080 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.800444534999997, 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.081 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:10:59.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:10:59.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.084 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.800411785999998, 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.085 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:10:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:10:59.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.800384893, 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:10:59.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:59.093 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.800359620000002, 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.800328493000002, 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.098 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.799685136999997, 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794499209999998, 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:10:59.106 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:10:59.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:10:59.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.110 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794477727, 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:10:59.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794453441000002, 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:10:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:10:59.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794435774, 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:10:59.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:10:59.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794412188999999, 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.123 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:10:59.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:10:59.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794396173999999, 2024-09-21T23:10:59.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:10:59.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.131 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794382343000002, 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:59.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:10:59.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:59.135 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794350981999997, 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:10:59.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:10:59.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.794282559999999, 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:10:59.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:10:59.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:10:59.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:10:59.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:10:59.144 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.144 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.144 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.144 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:10:59.144 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:10:59.144 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:10:59.145 INFO:teuthology.orchestra.run.smithi150.stdout:3527 read 0x1cb60e6 thru 0x1cbf0e6 (0x9001 bytes) 2024-09-21T23:10:59.145 INFO:teuthology.orchestra.run.smithi150.stdout:3528 punch from 0xd6edc8 to 0xd7a732, (0xb96a bytes) 2024-09-21T23:10:59.146 INFO:teuthology.orchestra.run.smithi150.stdout:3529 write 0xb0a7c61 thru 0xb0ab024 (0x33c4 bytes) 2024-09-21T23:10:59.146 INFO:teuthology.orchestra.run.smithi150.stdout:3530 trunc from 0xb0ab025 to 0x56f7d62 2024-09-21T23:10:59.146 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:59.143+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:59.146 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:10:59.152 INFO:teuthology.orchestra.run.smithi150.stdout:3531 read 0x2a1db7c thru 0x2a1fa9e (0x1f23 bytes) 2024-09-21T23:10:59.152 INFO:teuthology.orchestra.run.smithi150.stdout:3533 write 0x41410f5 thru 0x41425de (0x14ea bytes) 2024-09-21T23:10:59.158 INFO:teuthology.orchestra.run.smithi150.stdout:3534 read 0x2032de6 thru 0x203cef4 (0xa10f bytes) 2024-09-21T23:10:59.167 INFO:teuthology.orchestra.run.smithi175.stderr:no valid command found; 10 closest matches: 2024-09-21T23:10:59.167 INFO:teuthology.orchestra.run.smithi175.stderr:0 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:1 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:2 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:abort 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:assert 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:bluestore allocator dump block 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:bluestore allocator fragmentation block 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:bluestore allocator score block 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:config diff 2024-09-21T23:10:59.168 INFO:teuthology.orchestra.run.smithi175.stderr:admin_socket: invalid command 2024-09-21T23:10:59.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:10:59.170 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-21T23:10:59.182 INFO:teuthology.orchestra.run.smithi150.stdout:3536 write 0xa9ebe87 thru 0xa9efcfa (0x3e74 bytes) 2024-09-21T23:10:59.187 INFO:teuthology.orchestra.run.smithi150.stdout:3537 read 0x39d1dc4 thru 0x39d2f90 (0x11cd bytes) 2024-09-21T23:10:59.188 INFO:teuthology.orchestra.run.smithi150.stdout:3538 writesame 0x4ee0f74 thru 0x4eeb573 data_size 0x53(0xa600 bytes) 2024-09-21T23:10:59.192 INFO:teuthology.orchestra.run.smithi150.stdout:3540 writesame 0x910d45 thru 0x91433a data_size 0x1(0x35f6 bytes) 2024-09-21T23:10:59.205 INFO:teuthology.orchestra.run.smithi150.stdout:3541 read 0x373f80c thru 0x3744255 (0x4a4a bytes) 2024-09-21T23:10:59.205 INFO:teuthology.orchestra.run.smithi150.stdout:3544 writesame 0x8ed2ed3 thru 0x8ee180e data_size 0x76(0xe93c bytes) 2024-09-21T23:10:59.210 INFO:teuthology.orchestra.run.smithi150.stdout:3545 read 0x13be7e7 thru 0x13cbb4f (0xd369 bytes) 2024-09-21T23:10:59.211 INFO:teuthology.orchestra.run.smithi150.stdout:3546 read 0x5428e9f thru 0x5429ac4 (0xc26 bytes) 2024-09-21T23:10:59.211 INFO:teuthology.orchestra.run.smithi150.stdout:3549 read 0x96ac2a7 thru 0x96ad71b (0x1475 bytes) 2024-09-21T23:10:59.226 INFO:teuthology.orchestra.run.smithi150.stdout:3550 writesame 0xbbefc1e thru 0xbbfa660 data_size 0x57(0xaa43 bytes) 2024-09-21T23:10:59.246 INFO:teuthology.orchestra.run.smithi150.stdout:3551 trunc from 0xbbfa661 to 0x2f56701 2024-09-21T23:10:59.246 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:10:59.244+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:10:59.266 INFO:teuthology.orchestra.run.smithi150.stdout:3552 punch from 0x11603b6 to 0x1161e66, (0x1ab0 bytes) 2024-09-21T23:10:59.279 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:59.279 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:59.279 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:10:59.279 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:59.297 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:10:59.297 INFO:teuthology.orchestra.run.smithi150.stdout:3554 write 0xb87f279 thru 0xb885dc4 (0x6b4c bytes) 2024-09-21T23:10:59.301 INFO:teuthology.orchestra.run.smithi150.stdout:3556 writesame 0x3b1f0da thru 0x3b2b2b4 data_size 0x1(0xc1db bytes) 2024-09-21T23:10:59.346 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:59.343+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:59.348 INFO:teuthology.orchestra.run.smithi150.stdout:3558 punch from 0x5d8ff81 to 0x5d9119a, (0x1219 bytes) 2024-09-21T23:10:59.349 INFO:teuthology.orchestra.run.smithi150.stdout:3559 trunc from 0xb885dc5 to 0xafdb8b6 2024-09-21T23:10:59.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:10:59.360 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.361 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.361 INFO:teuthology.orchestra.run.smithi150.stdout:3563 punch from 0xd5ac49 to 0xd5f158, (0x450f bytes) 2024-09-21T23:10:59.361 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.361 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.361 INFO:teuthology.orchestra.run.smithi150.stdout:3567 writesame 0x27fa17d thru 0x28093c8 data_size 0x12(0xf24c bytes) 2024-09-21T23:10:59.364 INFO:teuthology.orchestra.run.smithi150.stdout:3568 read 0x4d44a51 thru 0x4d475bf (0x2b6f bytes) 2024-09-21T23:10:59.364 INFO:teuthology.orchestra.run.smithi150.stdout:3569 punch from 0x777de92 to 0x7787c9a, (0x9e08 bytes) 2024-09-21T23:10:59.372 INFO:teuthology.orchestra.run.smithi150.stdout:3571 trunc from 0xafdb8b6 to 0x65e2024 2024-09-21T23:10:59.388 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.388 INFO:teuthology.orchestra.run.smithi150.stdout:3575 read 0x5226199 thru 0x52277d4 (0x163c bytes) 2024-09-21T23:10:59.388 INFO:teuthology.orchestra.run.smithi150.stdout:3577 read 0x2ce959d thru 0x2ceac79 (0x16dd bytes) 2024-09-21T23:10:59.399 INFO:teuthology.orchestra.run.smithi150.stdout:3578 trunc from 0x65e2024 to 0xcbf99a4 2024-09-21T23:10:59.407 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.407 INFO:teuthology.orchestra.run.smithi150.stdout:3580 writesame 0x960b037 thru 0x960c521 data_size 0x2d(0x14eb bytes) 2024-09-21T23:10:59.412 INFO:teuthology.orchestra.run.smithi150.stdout:3581 read 0x6e39659 thru 0x6e4098a (0x7332 bytes) 2024-09-21T23:10:59.412 INFO:teuthology.orchestra.run.smithi150.stdout:3582 read 0xa8c15ad thru 0xa8d11c9 (0xfc1d bytes) 2024-09-21T23:10:59.412 INFO:teuthology.orchestra.run.smithi150.stdout:3583 read 0x21b481c thru 0x21b96c5 (0x4eaa bytes) 2024-09-21T23:10:59.413 INFO:teuthology.orchestra.run.smithi150.stdout:3584 writesame 0xca7ebcc thru 0xca85368 data_size 0x5(0x679d bytes) 2024-09-21T23:10:59.420 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.420 INFO:teuthology.orchestra.run.smithi150.stdout:3586 punch from 0xa1d17ae to 0xa1d32ea, (0x1b3c bytes) 2024-09-21T23:10:59.422 INFO:teuthology.orchestra.run.smithi150.stdout:3588 trunc from 0xcbf99a4 to 0xdbecc66 2024-09-21T23:10:59.429 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.430 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.430 INFO:teuthology.orchestra.run.smithi150.stdout:3593 punch from 0x8b0107f to 0x8b10493, (0xf414 bytes) 2024-09-21T23:10:59.430 INFO:teuthology.orchestra.run.smithi150.stdout:3594 read 0x9751b7 thru 0x978376 (0x31c0 bytes) 2024-09-21T23:10:59.436 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:10:59.436 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:10:59.436 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:10:59.436 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:10:59.437 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:10:59.437 INFO:teuthology.orchestra.run.smithi150.stdout:3597 trunc from 0xdbecc66 to 0xa453a00 2024-09-21T23:10:59.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:10:59.447 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:59.447+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:10:59.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:10:59.455 INFO:teuthology.orchestra.run.smithi150.stdout:3599 writesame 0x3106b49 thru 0x310ac47 data_size 0x1(0x40ff bytes) 2024-09-21T23:10:59.475 INFO:teuthology.orchestra.run.smithi150.stdout:3600 read 0x9404efb thru 0x940bc8c (0x6d92 bytes) 2024-09-21T23:10:59.481 INFO:teuthology.orchestra.run.smithi150.stdout:3601 writesame 0xa625e7b thru 0xa630767 data_size 0x9(0xa8ed bytes) 2024-09-21T23:10:59.492 INFO:teuthology.orchestra.run.smithi150.stdout:3602 trunc from 0xa630768 to 0x4fbea24 2024-09-21T23:10:59.506 INFO:teuthology.orchestra.run.smithi150.stdout:3603 read 0x3c46e2c thru 0x3c47f77 (0x114c bytes) 2024-09-21T23:10:59.506 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:10:59.527 INFO:teuthology.orchestra.run.smithi150.stdout:3605 write 0xc8cfe21 thru 0xc8d5a6f (0x5c4f bytes) 2024-09-21T23:10:59.532 INFO:teuthology.orchestra.run.smithi150.stdout:3606 write 0x9de6c19 thru 0x9df2331 (0xb719 bytes) 2024-09-21T23:10:59.537 INFO:teuthology.orchestra.run.smithi150.stdout:3607 write 0x25c27e6 thru 0x25caaae (0x82c9 bytes) 2024-09-21T23:10:59.540 INFO:teuthology.orchestra.run.smithi150.stdout:3608 write 0x6d6da3a thru 0x6d76400 (0x89c7 bytes) 2024-09-21T23:10:59.544 INFO:teuthology.orchestra.run.smithi150.stdout:3611 clone 29 order 24 su 4194304 sc 3 2024-09-21T23:10:59.548 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:59.547+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:59.598 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:10:59.598 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:10:59.614 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:10:59.615 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:10:59.627 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:59.627 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:59.631 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:10:59.631 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:10:59.646 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:10:59.649 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:59.647+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:59.651 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:10:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:10:59.652 INFO:teuthology.orchestra.run.smithi150.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T23:10:59.673 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:10:59.673 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:10:59.693 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:10:59.694 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:10:59.714 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:10:59.714 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:10:59.718 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:10:59.718 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:10:59.749 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:10:59.747+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:10:59.766 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:59.767 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:59.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:10:59.851 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:10:59.851+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:10:59.901 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:10:59.901 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:10:59.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:10:59.952 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:10:59.951+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:00.037 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:00.037 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:00.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:00.052 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:00.052+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:00.111 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:00.111+0000 7fdddc67c7c0 -1 osd.4 32 log_to_monitors true 2024-09-21T23:11:00.154 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:00.151+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:00.178 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:00.178 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:00.193 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:00.254 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:00.251+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:00.315 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:00.315 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:00.329 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:00.343 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:11:00.356 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:00.352+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:00.380 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:00.379+0000 7fddd0449640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:11:00.426 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone28 from 0xc8d5a70 (overlap 0x275096) to 0x0 2024-09-21T23:11:00.456 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:00.455+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:00.462 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:00.462 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:00.480 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:00.526 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-21T23:11:00.556 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:00.556+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:00.605 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:00.605 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:00.622 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:00.657 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:00.655+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:00.686 INFO:teuthology.orchestra.run.smithi150.stdout:3613 trunc from 0xc8d5a70 to 0x509fa47 2024-09-21T23:11:00.712 INFO:teuthology.orchestra.run.smithi150.stdout:3614 write 0x3cd86d0 thru 0x3ce685c (0xe18d bytes) 2024-09-21T23:11:00.724 INFO:teuthology.orchestra.run.smithi150.stdout:3615 writesame 0x6d370e6 thru 0x6d3ee9c data_size 0x1(0x7db7 bytes) 2024-09-21T23:11:00.724 INFO:teuthology.orchestra.run.smithi150.stdout:3616 read 0x21e37a7 thru 0x21e5a86 (0x22e0 bytes) 2024-09-21T23:11:00.725 INFO:teuthology.orchestra.run.smithi150.stdout:3617 read 0x13228ac thru 0x1331223 (0xe978 bytes) 2024-09-21T23:11:00.726 INFO:teuthology.orchestra.run.smithi150.stdout:3618 read 0x3a1eafe thru 0x3a237b7 (0x4cba bytes) 2024-09-21T23:11:00.726 INFO:teuthology.orchestra.run.smithi150.stdout:3619 write 0x23a9566 thru 0x23b4bac (0xb647 bytes) 2024-09-21T23:11:00.736 INFO:teuthology.orchestra.run.smithi150.stdout:3620 write 0x48768e2 thru 0x487c59b (0x5cba bytes) 2024-09-21T23:11:00.756 INFO:teuthology.orchestra.run.smithi150.stdout:3621 write 0xc5ad1f6 thru 0xc5bd0c0 (0xfecb bytes) 2024-09-21T23:11:00.758 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:00.755+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:00.763 INFO:teuthology.orchestra.run.smithi150.stdout:3622 write 0x2b72a83 thru 0x2b81421 (0xe99f bytes) 2024-09-21T23:11:00.773 INFO:teuthology.orchestra.run.smithi150.stdout:3624 writesame 0x82b87b2 thru 0x82c4809 data_size 0x14(0xc058 bytes) 2024-09-21T23:11:00.781 INFO:teuthology.orchestra.run.smithi150.stdout:3626 write 0x6ded1d0 thru 0x6dfac31 (0xda62 bytes) 2024-09-21T23:11:00.787 INFO:teuthology.orchestra.run.smithi150.stdout:3630 read 0x540204d thru 0x540a7cc (0x8780 bytes) 2024-09-21T23:11:00.788 INFO:teuthology.orchestra.run.smithi150.stdout:3631 read 0x6e9d64f thru 0x6ea58a2 (0x8254 bytes) 2024-09-21T23:11:00.789 INFO:teuthology.orchestra.run.smithi150.stdout:3632 punch from 0x1559514 to 0x15598be, (0x3aa bytes) 2024-09-21T23:11:00.789 INFO:teuthology.orchestra.run.smithi150.stdout:3633 punch from 0x527e965 to 0x5284a09, (0x60a4 bytes) 2024-09-21T23:11:00.789 INFO:teuthology.orchestra.run.smithi150.stdout:3634 read 0x164ecc7 thru 0x164f569 (0x8a3 bytes) 2024-09-21T23:11:00.790 INFO:teuthology.orchestra.run.smithi150.stdout:3635 trunc from 0xc5bd0c1 to 0xc1c30a9 2024-09-21T23:11:00.798 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:00.798 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:00.808 INFO:teuthology.orchestra.run.smithi150.stdout:3636 write 0x54ee319 thru 0x54fb465 (0xd14d bytes) 2024-09-21T23:11:00.812 INFO:teuthology.orchestra.run.smithi150.stdout:3638 write 0xa5301d3 thru 0xa5310e7 (0xf15 bytes) 2024-09-21T23:11:00.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:00.818 INFO:teuthology.orchestra.run.smithi150.stdout:3639 read 0x16ab652 thru 0x16b7815 (0xc1c4 bytes) 2024-09-21T23:11:00.818 INFO:teuthology.orchestra.run.smithi150.stdout:3640 clone 30 order 21 su 1048576 sc 6 2024-09-21T23:11:00.864 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:00.864+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:00.935 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:00.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:00.992 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:00.991+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:01.061 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:01.059+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:01.081 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:01.081 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:01.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:01.162 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:01.159+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:01.263 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:01.259+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:01.263 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:01.263 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:01.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:01.364 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:01.363+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:01.408 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:01.409 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:01.423 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:01.464 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:01.463+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:01.565 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:01.563+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:01.586 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:01.586 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:01.600 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:01.666 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:01.663+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:01.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:01.765 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:01.765 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:01.766 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:01.763+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:01.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:01.871 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:01.871+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:01.876 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:01.877 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:01.893 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:01.893 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:01.913 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:01.913 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:01.931 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:01.931 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:01.945 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:01.945 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:01.951 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:01.951 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-21T23:11:01.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:01.968 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:01.967+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:01.971 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:01.971 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:01.989 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:01.989 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:02.008 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:02.008 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:02.069 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:02.067+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:02.125 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:02.125 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:02.125 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:02.125 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:02.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:02.170 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:02.167+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:02.271 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:02.267+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:02.306 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:02.306 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:02.320 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:02.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:11:02.371 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:02.371+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.258912346000002, 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.443 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:11:02.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.25880926, 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:02.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:11:02.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:11:02.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.258741911000001, 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:02.451 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:11:02.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:11:02.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.183554975, 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:11:02.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.183499206999997, 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.459 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.183467667999999, 2024-09-21T23:11:02.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:11:02.464 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.464 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.464 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.465 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.465 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.465 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.465 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:11:02.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.465 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:11:02.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.183434919, 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.469 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:11:02.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.183408026000002, 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:11:02.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.183382752999997, 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:11:02.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:11:02.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.481 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.183351625999997, 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:11:02.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.485 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.182708269999999, 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:11:02.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:11:02.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177522343, 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.490 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:11:02.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:11:02.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:11:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177500860000002, 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:11:02.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:11:02.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177476574000003, 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.498 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:11:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177458907000002, 2024-09-21T23:11:02.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:02.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177435322000001, 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:11:02.506 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:11:02.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177419307000001, 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:11:02.510 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:02.514 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177405475999997, 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:11:02.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:11:02.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.518 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177374114999999, 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:02.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:11:02.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:11:02.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 55.177305693000001, 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:02.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:11:02.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:11:02.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:02.526 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:02.528 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:02.528 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:02.523+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:02.629 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:02.627+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:02.642 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:02.642 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:02.642 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:02.656 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:02.730 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:02.727+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:02.771 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:02.771 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:02.785 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:02.873 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:02.871+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:02.932 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:02.927+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.346498971999999, 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:11:02.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.346452305, 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:11:02.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:11:02.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:11:02.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:11:02.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.346378239, 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.346315513, 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:11:02.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.346263563000001, 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.968 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:11:02.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:11:02.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.972 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.346050476999999, 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.976 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.344166452, 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.344118401999999, 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:11:02.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.344070094999999, 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.985 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:11:02.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:11:02.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.344022880000001, 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:11:02.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:11:02.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.343958651000001, 2024-09-21T23:11:02.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.343906741000001, 2024-09-21T23:11:02.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:02.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40186 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 442368~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.339359138999999, 2024-09-21T23:11:03.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093234290999999997, 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40186 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610104+0000", 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610109+0000", 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2140000000000004e-06 2024-09-21T23:11:03.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610125+0000", 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5276000000000001e-05 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610139+0000", 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4477e-05 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703148+0000", 2024-09-21T23:11:03.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093008570999999998 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703236+0000", 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7868000000000006e-05 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703337+0000", 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101742 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.339304447, 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:11:03.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.339249632000001, 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:11:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.339201908, 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.339135019, 2024-09-21T23:11:03.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:03.022 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.335589741, 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.335543000000001, 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:11:03.026 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:03.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:11:03.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.335496478, 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.034 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:03.035 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:03.035 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:03.031+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:03.136 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:03.135+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:03.203 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.83651879600000001, 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.83656464200000002, 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for peered", 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:11:03.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:11:03.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.208 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.208 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:11:03.208 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 1 2024-09-21T23:11:03.208 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:03.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:03.268 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:03.267+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:03.338 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:03.335+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:03.339 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:03.340 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:03.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:03.366 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:11:03.440 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:03.439+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:03.504 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:03.504 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:03.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:03.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:03.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51023 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265705628999999, 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037458331999999997, 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51023 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235893+0000", 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5100000000000001e-07 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235895+0000", 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3240000000000001e-06 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235901+0000", 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0610000000000001e-06 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235906+0000", 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1830000000000002e-06 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272599+0000", 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036692726000000002 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272669+0000", 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9417000000000003e-05 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273351+0000", 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00068227000000000001 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.508 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51024 3.4 3:2264aa58:::rbd_header.1149796cbcee:head [watch reconnect cookie 139807362363424 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265685768000001, 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039816437000000003, 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51024 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.509 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235913+0000", 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235915+0000", 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.897e-06 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235921+0000", 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3759999999999996e-06 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235926+0000", 2024-09-21T23:11:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7759999999999997e-06 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273380+0000", 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037453792999999999 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273460+0000", 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0241999999999995e-05 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275729+0000", 2024-09-21T23:11:03.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022688740000000002 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51026 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265666297999999, 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038933657000000003, 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51026 2024-09-21T23:11:03.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.41e-07 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235935+0000", 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6369999999999999e-06 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235941+0000", 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.528e-06 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235947+0000", 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2959999999999998e-06 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274454+0000", 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038507844999999999 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274533+0000", 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.839e-05 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274865+0000", 2024-09-21T23:11:03.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033262 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51028 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265645511000001, 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03952514, 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51028 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235955+0000", 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0640000000000001e-06 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235961+0000", 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1099999999999999e-06 2024-09-21T23:11:03.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235966+0000", 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4120000000000003e-06 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274896+0000", 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038930017999999997 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274973+0000", 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7616999999999999e-05 2024-09-21T23:11:03.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275478+0000", 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000504544 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51030 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265626721, 2024-09-21T23:11:03.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040194278, 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51030 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235972+0000", 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235974+0000", 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3159999999999999e-06 2024-09-21T23:11:03.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235980+0000", 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235985+0000", 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0359999999999997e-06 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275761+0000", 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776211999999998 2024-09-21T23:11:03.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275838+0000", 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7231999999999995e-05 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276166+0000", 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032743300000000002 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51056 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265525372000001, 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040641749999999997, 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51056 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236080+0000", 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0180000000000004e-06 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236087+0000", 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8759999999999999e-06 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236092+0000", 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276196+0000", 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040103449999999999 2024-09-21T23:11:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276272+0000", 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6062000000000005e-05 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276715+0000", 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000442719 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51057 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265498807, 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041665445000000002, 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51057 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236100+0000", 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236102+0000", 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0990000000000001e-06 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236110+0000", 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1610000000000003e-06 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236115+0000", 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9880000000000004e-06 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276744+0000", 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040628876000000001 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276810+0000", 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6359999999999995e-05 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277765+0000", 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00095440200000000003 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51058 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265476253999999, 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041431192999999998, 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51058 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:03.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236125+0000", 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236131+0000", 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4489999999999996e-06 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236136+0000", 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277067+0000", 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040930381000000002 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277212+0000", 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014488599999999999 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277553+0000", 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000341424 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51059 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.265455533000001, 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042250247999999997, 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51059 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236145+0000", 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236151+0000", 2024-09-21T23:11:03.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.268e-06 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236157+0000", 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4480000000000002e-06 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277579+0000", 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041421948 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277693+0000", 2024-09-21T23:11:03.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113786 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.278393+0000", 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00070025299999999996 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51324 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:03.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.248743655, 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037384598999999998, 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51324 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:03.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252855+0000", 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252857+0000", 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2709999999999999e-06 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252864+0000", 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6889999999999998e-06 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252869+0000", 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1429999999999997e-06 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290081+0000", 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037211602000000003 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290154+0000", 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3200000000000004e-05 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290239+0000", 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5191000000000005e-05 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51325 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.248722225, 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037730817, 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51325 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252879+0000", 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2239999999999998e-06 2024-09-21T23:11:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252885+0000", 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3010000000000003e-06 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252890+0000", 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3569999999999997e-06 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290267+0000", 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037376615000000002 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290338+0000", 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1265000000000003e-05 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290607+0000", 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026868200000000001 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51338 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.248618314, 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038044604000000003, 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.552 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51338 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8099999999999998e-07 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252982+0000", 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3020000000000002e-06 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252989+0000", 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3149999999999997e-06 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252994+0000", 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9359999999999998e-06 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290639+0000", 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037645064999999998 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290713+0000", 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4115000000000004e-05 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291024+0000", 2024-09-21T23:11:03.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031148999999999999 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51339 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.248596869, 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038794718999999998, 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51339 2024-09-21T23:11:03.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253002+0000", 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:03.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253005+0000", 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253011+0000", 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4949999999999996e-06 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253016+0000", 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:11:03.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291053+0000", 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038037165999999997 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291126+0000", 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2575000000000002e-05 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291796+0000", 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067037100000000001 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.559 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51340 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.248562401999999, 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039295921999999997, 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51340 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9699999999999996e-07 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253039+0000", 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5220000000000002e-06 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253046+0000", 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253051+0000", 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1869999999999996e-06 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291828+0000", 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038776772000000001 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291914+0000", 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5847999999999995e-05 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292332+0000", 2024-09-21T23:11:03.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000418136 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51341 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.248540393000001, 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039551238000000002, 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51341 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253064+0000", 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253072+0000", 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0339999999999997e-06 2024-09-21T23:11:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253077+0000", 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.921e-06 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292363+0000", 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039285863999999997 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292437+0000", 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4247999999999998e-05 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292609+0000", 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017169900000000001 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51475 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.245639390999999, 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037285393999999999, 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51475 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255960+0000", 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255965+0000", 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8760000000000004e-06 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255977+0000", 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2629000000000001e-05 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255985+0000", 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1179999999999992e-06 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292638+0000", 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036653036 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292713+0000", 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4262000000000002e-05 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293244+0000", 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00053145700000000004 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51477 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.245598332, 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037740821000000001, 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51477 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9599999999999999e-07 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256003+0000", 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.627e-06 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256011+0000", 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256016+0000", 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.784e-06 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293276+0000", 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037260003 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293363+0000", 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6558999999999996e-05 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293741+0000", 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037767200000000001 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51479 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.245573695999999, 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038089583000000003, 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51479 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256025+0000", 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.13e-07 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256027+0000", 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1399999999999998e-06 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256034+0000", 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256038+0000", 2024-09-21T23:11:03.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293776+0000", 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037737241999999997 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293852+0000", 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6287999999999999e-05 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294114+0000", 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026198699999999998 2024-09-21T23:11:03.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51481 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.245552666, 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038735549000000001, 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51481 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256046+0000", 2024-09-21T23:11:03.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256048+0000", 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2359999999999999e-06 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256055+0000", 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6669999999999999e-06 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256061+0000", 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6200000000000004e-06 2024-09-21T23:11:03.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294146+0000", 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038085073999999997 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294234+0000", 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8839999999999996e-05 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294781+0000", 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00054657999999999996 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.583 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.133882866, 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.584 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:11:03.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:03.588 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:03.589 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone29 from 0xc1c30a9 (overlap 0x509fa47) to 0x80ff6d 2024-09-21T23:11:03.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:03.590 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:03.587+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:03.615 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-21T23:11:03.618 INFO:teuthology.orchestra.run.smithi150.stdout:3641 punch from 0xb4588c3 to 0xb467f42, (0xf67f bytes) 2024-09-21T23:11:03.623 INFO:teuthology.orchestra.run.smithi150.stdout:3643 punch from 0x75299d4 to 0x7530300, (0x692c bytes) 2024-09-21T23:11:03.623 INFO:teuthology.orchestra.run.smithi150.stdout:3644 write 0x1da760a thru 0x1dae508 (0x6eff bytes) 2024-09-21T23:11:03.632 INFO:teuthology.orchestra.run.smithi150.stdout:3645 read 0x7ac2ac3 thru 0x7ad13c2 (0xe900 bytes) 2024-09-21T23:11:03.633 INFO:teuthology.orchestra.run.smithi150.stdout:3646 write 0x92576e7 thru 0x9261261 (0x9b7b bytes) 2024-09-21T23:11:03.637 INFO:teuthology.orchestra.run.smithi150.stdout:3647 read 0xb0e57d2 thru 0xb0e8424 (0x2c53 bytes) 2024-09-21T23:11:03.638 INFO:teuthology.orchestra.run.smithi150.stdout:3649 writesame 0x6a941a7 thru 0x6aa178c data_size 0x26(0xd5e6 bytes) 2024-09-21T23:11:03.645 INFO:teuthology.orchestra.run.smithi150.stdout:3650 writesame 0x8313a07 thru 0x8319db7 data_size 0x2f(0x63b1 bytes) 2024-09-21T23:11:03.651 INFO:teuthology.orchestra.run.smithi150.stdout:3651 punch from 0x7c898c9 to 0x7c8cb30, (0x3267 bytes) 2024-09-21T23:11:03.651 INFO:teuthology.orchestra.run.smithi150.stdout:3652 clone 31 order 18 su 16384 sc 15 2024-09-21T23:11:03.690 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:03.687+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:03.742 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:03.742 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:03.757 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:03.791 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:03.787+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:03.892 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:03.891+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:03.923 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:03.923 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:03.923 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:03.924 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:03.924 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:03.938 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:03.992 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:03.991+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:04.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:04.064 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:04.064 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:04.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:04.064 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:04.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.571066255000002, 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:11:04.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.068 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.571012817, 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.069 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:11:04.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:11:04.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.072 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.570961841999999, 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.073 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:11:04.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.570910910999999, 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:11:04.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:11:04.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:11:04.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.570855600000002, 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:11:04.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.570786685000002, 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:04.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:11:04.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:11:04.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.566149331999998, 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.089 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:04.093 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.566081190999999, 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.097 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.079927680000001, 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:11:04.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:04.101 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.078874304999999, 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:11:04.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.078827938, 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:11:04.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:11:04.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.078699758999999, 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.078613963999999, 2024-09-21T23:11:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:04.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.078070148999998, 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.077972369000001, 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:11:04.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:11:04.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.126 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.077897494999998, 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:11:04.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.130 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.077836717, 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:04.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:11:04.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:04.134 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.077800399000001, 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:11:04.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.273338604999999, 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:04.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.756800439, 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.143 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:11:04.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:11:04.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:04.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.147 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:04.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:04.148 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:04.149 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:04.149 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:04.147+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:04.172 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:04.176 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:04.177 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:04.195 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:04.195 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:04.213 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:04.213 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:04.230 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:04.230 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:04.249 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:04.247+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:04.252 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:04.252 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:04.276 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:04.292 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:04.292 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:04.295 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:04.295 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:04.307 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:04.318 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:04.318 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:04.319 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:04.319 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:04.375 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:04.375+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:04.449 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:04.450 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:04.451 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:04.447+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:04.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:04.552 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:04.551+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:04.628 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:04.628 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:04.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:04.684 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:04.683+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:04.754 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:04.751+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:04.773 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:04.773 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:04.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:04.855 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:04.851+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:04.906 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:04.906 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:04.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:04.964 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:04.963+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:05.056 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:05.055+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:05.092 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:05.092 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:05.107 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:05.157 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:05.155+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:05.233 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:05.233 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:05.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:05.295 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:05.295+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:05.359 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:05.355+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:05.377 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:05.377 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:05.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:11:05.400 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:05.459 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:05.455+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:05.560 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:05.559+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:05.578 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:05.578 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:05.595 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:05.661 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:05.659+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:05.726 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:05.726 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:05.740 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:05.762 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:05.759+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:05.863 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:05.859+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:05.926 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:05.926 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:05.943 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:05.963 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:05.959+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:06.064 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:06.063+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:06.069 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:06.070 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:06.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:06.165 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:06.163+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:06.266 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:06.263+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:06.274 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:06.274 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:06.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:06.336 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:06.370 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:11:06.370 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:06.367+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:06.461 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:06.461 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:06.467 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:06.467+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:06.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:06.499 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:06.499 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:06.516 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:06.516 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:06.532 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:06.533 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:06.549 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:06.549 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:06.568 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:06.568 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:06.568 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:06.567+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:06.585 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:06.586 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:06.604 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:06.604 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:06.622 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:06.622 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2534031859999999, 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:06.646 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:11:06.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:11:06.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.e 71/70 [PushOp(3:74244987:::rbd_data.1149e7c0ffc2.000000000000001b:9, version: 34'39, data_included: [0~1015808], data_size: 1015808, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:74244987:::rbd_data.1149e7c0ffc2.000000000000001b:9@34'39, size: 1015808, copy_subset: [(0, 1015808)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1015808, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.418685+0000", 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.2249221119999998, 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098218360000000005, 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.418685+0000", 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.418685+0000", 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.418687+0000", 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.069e-06 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.420732+0000", 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020448739999999999 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.420786+0000", 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3961000000000001e-05 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.420869+0000", 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2705000000000002e-05 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.421015+0000", 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014644699999999999 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.421047+0000", 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1661000000000001e-05 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.428507+0000", 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0074601190000000003 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.654 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.e 71/70 [PushOp(3:71512ace:::rbd_data.1149c115bf92.0000000000000001:d, version: 38'76, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:71512ace:::rbd_data.1149c115bf92.0000000000000001:d@38'76, size: 12396774, copy_subset: [(0, 12396774)], clone_subset: {}, snapset: d=[]:{d: [d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.467045+0000", 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.1765625119999998, 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003462536, 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.467045+0000", 2024-09-21T23:11:06.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.467045+0000", 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.467046+0000", 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.012e-06 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.468144+0000", 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001098173 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.468176+0000", 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1778e-05 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.468206+0000", 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9672e-05 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.468405+0000", 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000198949 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.468436+0000", 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1248999999999998e-05 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.470507+0000", 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020717029999999998 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.658 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.4425.0:61396 3.1b e71/70 3:da25b0d9:::rbd_data.114950b78073.0000000000000000:head v 71'278, mlcod=0'0)", 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.533927+0000", 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.1096804750000002, 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0083382960000000002, 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.533927+0000", 2024-09-21T23:11:06.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.533927+0000", 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.533927+0000", 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8099999999999998e-07 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.536470+0000", 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025425629999999999 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.536488+0000", 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7931000000000002e-05 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.536497+0000", 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8929999999999994e-06 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.536557+0000", 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0142000000000001e-05 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.536581+0000", 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3762000000000001e-05 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.542220+0000", 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056391779999999999 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.542265+0000", 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5445999999999998e-05 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.4425.0:61397 3.1b e71/70 3:da25b0d9:::rbd_data.114950b78073.0000000000000000:head v 71'280, mlcod=0'0)", 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.537284+0000", 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.1063237209999999, 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0064372709999999996, 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.537284+0000", 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.537284+0000", 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.537286+0000", 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9230000000000001e-06 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.537302+0000", 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6345999999999999e-05 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.537332+0000", 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0239e-05 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.537351+0000", 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9066e-05 2024-09-21T23:11:06.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.539708+0000", 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023571059999999999 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.539734+0000", 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5341999999999999e-05 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.543690+0000", 2024-09-21T23:11:06.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0039558800000000002 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.543721+0000", 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1368999999999997e-05 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.e 71/70 [PushOp(3:7020e7dc:::rbd_data.1149a17d43fd.0000000000000007:head, version: 49'143, data_included: [0~917504], data_size: 917504, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:7020e7dc:::rbd_data.1149a17d43fd.0000000000000007:head@49'143, size: 917504, copy_subset: [(0, 917504)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 917504, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.547741+0000", 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.0958663409999998, 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0040016899999999996, 2024-09-21T23:11:06.667 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.547741+0000", 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.547741+0000", 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.547747+0000", 2024-09-21T23:11:06.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6030000000000004e-06 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.549661+0000", 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019138009999999999 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.549707+0000", 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6239000000000002e-05 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.549738+0000", 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1102000000000001e-05 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.549932+0000", 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019371300000000001 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.549954+0000", 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.245e-05 2024-09-21T23:11:06.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.551743+0000", 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001788782 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.f 71/70 [PushOp(3:f5b9d2b6:::rbd_data.1149e7c0ffc2.000000000000000a:9, version: 34'204, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f5b9d2b6:::rbd_data.1149e7c0ffc2.000000000000000a:9@34'204, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.605170+0000", 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.038437579, 2024-09-21T23:11:06.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025805559999999999, 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.605170+0000", 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.605170+0000", 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.605171+0000", 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4e-07 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.606234+0000", 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010630749999999999 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.606258+0000", 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4644e-05 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.606274+0000", 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6030000000000001e-05 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.606377+0000", 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102036 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.606388+0000", 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1188e-05 2024-09-21T23:11:06.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.607750+0000", 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001362743 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.4425.0:61489 3.1a e71/70 3:5db6ab8b:::rbd_data.1149904dce8.0000000000000011:head v 71'808, mlcod=34'93)", 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.630777+0000", 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.0128308939999999, 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002687329, 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.675 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.630777+0000", 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.630777+0000", 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.630777+0000", 2024-09-21T23:11:06.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.630843+0000", 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5764000000000003e-05 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.630856+0000", 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3183000000000001e-05 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.630864+0000", 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4989999999999997e-06 2024-09-21T23:11:06.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.631007+0000", 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014369099999999999 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.631109+0000", 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010170000000000001 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.633406+0000", 2024-09-21T23:11:06.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022969209999999999 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.633464+0000", 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7923000000000001e-05 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.4425.0:61497 3.a e71/70 3:55f0f250:::rbd_data.1149904dce8.0000000000000034:head v 71'258, mlcod=36'25)", 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.644226+0000", 2024-09-21T23:11:06.679 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.9993818010000002, 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019730390000000002, 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644226+0000", 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644226+0000", 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644227+0000", 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6500000000000001e-06 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644271+0000", 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3574e-05 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644302+0000", 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1213000000000001e-05 2024-09-21T23:11:06.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644323+0000", 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1212e-05 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644524+0000", 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018999999999999 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.644594+0000", 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0563999999999997e-05 2024-09-21T23:11:06.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.646083+0000", 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014888 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.646199+0000", 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011583599999999999 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.683 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.f 71/70 [PushOp(3:f2041178:::rbd_data.11494fe4f7e7.000000000000000d:head, version: 56'335, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:f2041178:::rbd_data.11494fe4f7e7.000000000000000d:head@56'335, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.770955+0000", 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.872652881, 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021637230000000002, 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.770955+0000", 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.770955+0000", 2024-09-21T23:11:06.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.770956+0000", 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7069999999999999e-06 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.771535+0000", 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057853600000000005 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.771581+0000", 2024-09-21T23:11:06.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6264999999999998e-05 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.771615+0000", 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3602000000000001e-05 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.771798+0000", 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018318699999999999 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.771821+0000", 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2846000000000001e-05 2024-09-21T23:11:06.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.773118+0000", 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00129758 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.1d 71/70 [PushOp(3:bd01ef2d:::rbd_data.11494fe4f7e7.000000000000000f:1a, version: 57'179, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:bd01ef2d:::rbd_data.11494fe4f7e7.000000000000000f:1a@57'179, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 1a=[]:{1a: [1a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:03.968426+0000", 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.6751813709999999, 2024-09-21T23:11:06.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002573208, 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.968426+0000", 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.968426+0000", 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.968428+0000", 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7099999999999999e-06 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.968776+0000", 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034865699999999999 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.968837+0000", 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0897999999999999e-05 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.968873+0000", 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5815000000000002e-05 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.969106+0000", 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023295499999999999 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.969144+0000", 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:03.970999+0000", 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001855064 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.1d 71/70 [PushOp(3:ba2d9403:::rbd_data.1149796cbcee.0000000000000009:1d, version: 66'226, data_included: [0~196608], data_size: 196608, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ba2d9403:::rbd_data.1149796cbcee.0000000000000009:1d@66'226, size: 196608, copy_subset: [(0, 196608)], clone_subset: {}, snapset: 1d=[]:{1d: [1d]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 196608, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.045028+0000", 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.5985789910000001, 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019763279999999999, 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.691 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045028+0000", 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045028+0000", 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045030+0000", 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6199999999999999e-06 2024-09-21T23:11:06.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045165+0000", 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530899999999999 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045213+0000", 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045246+0000", 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3633000000000001e-05 2024-09-21T23:11:06.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045469+0000", 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022268900000000001 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.045507+0000", 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8352000000000001e-05 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.047005+0000", 2024-09-21T23:11:06.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001497401 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.1d 71/70 [PushOp(3:bf8f2834:::rbd_data.114952cb5a61.0000000000000000:1f, version: 68'254, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:bf8f2834:::rbd_data.114952cb5a61.0000000000000000:1f@68'254, size: 12582912, copy_subset: [(0, 12582912)], clone_subset: {}, snapset: 1f=[]:{1f: [1f]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.064542+0000", 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.579065505, 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037521260000000002, 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.064542+0000", 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.064542+0000", 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.064544+0000", 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6029999999999999e-06 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.065682+0000", 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001138703 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.065730+0000", 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8183000000000003e-05 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.065763+0000", 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2685000000000003e-05 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.066003+0000", 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000239371 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.066042+0000", 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9459000000000003e-05 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.068294+0000", 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022521220000000001 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.15 71/70 [PushOp(3:af3257c7:::rbd_data.1149a1629cf6.0000000000000006:head, version: 34'92, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:af3257c7:::rbd_data.1149a1629cf6.0000000000000006:head@34'92, size: 4980736, copy_subset: [(0, 4980736)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.102521+0000", 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.5410864640000002, 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037556059999999999, 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.102521+0000", 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.102521+0000", 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.102523+0000", 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6169999999999999e-06 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.104227+0000", 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017048199999999999 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.104274+0000", 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6916000000000003e-05 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.104305+0000", 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0289000000000001e-05 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.104523+0000", 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218412 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.104546+0000", 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3019000000000001e-05 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.106277+0000", 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001730533 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.15 71/70 [PushOp(3:af3257c7:::rbd_data.1149a1629cf6.0000000000000006:head, version: 34'92, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:af3257c7:::rbd_data.1149a1629cf6.0000000000000006:head@34'92, size: 4980736, copy_subset: [(0, 4980736)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.108976+0000", 2024-09-21T23:11:06.702 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.534631868, 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042290590000000003, 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.108976+0000", 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.108976+0000", 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.108977+0000", 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4300000000000001e-06 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.110446+0000", 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001469452 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.110520+0000", 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3218000000000003e-05 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.110584+0000", 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4307000000000006e-05 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.110809+0000", 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022510299999999999 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.110838+0000", 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8815000000000001e-05 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.113205+0000", 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023667340000000001 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.15 71/70 [PushOp(3:af3257c7:::rbd_data.1149a1629cf6.0000000000000006:head, version: 34'92, data_included: [2097152~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:af3257c7:::rbd_data.1149a1629cf6.0000000000000006:head@34'92, size: 4980736, copy_subset: [(0, 4980736)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4980736, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.115515+0000", 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.5280926419999998, 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026267500000000002, 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.706 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.115515+0000", 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.115515+0000", 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.115517+0000", 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7799999999999999e-06 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.116560+0000", 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010438400000000001 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.116607+0000", 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6195000000000001e-05 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.116639+0000", 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2353e-05 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.116836+0000", 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000196839 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.116852+0000", 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6200000000000001e-05 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.118142+0000", 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001289543 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.15 71/70 [PushOp(3:aab5b979:::rbd_data.1149f84cbfe6.0000000000000009:c, version: 37'122, data_included: [0~248742], data_size: 248742, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:aab5b979:::rbd_data.1149f84cbfe6.0000000000000009:c@37'122, size: 248742, copy_subset: [(0, 248742)], clone_subset: {}, snapset: c=[]:{c: [c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 248742, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.156056+0000", 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.4875511989999999, 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025216320000000002, 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.710 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156056+0000", 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156056+0000", 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156058+0000", 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.393e-06 2024-09-21T23:11:06.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156451+0000", 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039302299999999999 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156493+0000", 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2808000000000001e-05 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156527+0000", 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3512e-05 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156857+0000", 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000329859 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.156893+0000", 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6349e-05 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.158578+0000", 2024-09-21T23:11:06.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016846879999999999 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.15 71/70 [PushOp(3:aede9315:::rbd_data.11499b8b109.0000000000000003:10, version: 42'167, data_included: [1441866~55378], data_size: 55378, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:aede9315:::rbd_data.11499b8b109.0000000000000003:10@42'167, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 8388608, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.185686+0000", 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.4579211349999999, 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020888679999999998, 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.185686+0000", 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.185686+0000", 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.185688+0000", 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6440000000000001e-06 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.185755+0000", 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7306999999999998e-05 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.185803+0000", 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8012e-05 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.185875+0000", 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1596999999999999e-05 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.186106+0000", 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000231442 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.186127+0000", 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0956000000000001e-05 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.187775+0000", 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00164791 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.15 71/70 [PushOp(3:af0b784c:::rbd_data.11499b8b109.0000000000000008:10, version: 42'169, data_included: [143254~31434,8658136~21112], data_size: 52546, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:af0b784c:::rbd_data.11499b8b109.0000000000000008:10@42'169, size: 8679248, copy_subset: [(0, 8679248)], clone_subset: {}, snapset: 10=[]:{10: [10]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 8679248, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.191956+0000", 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.4516518700000001, 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019908080000000002, 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.191956+0000", 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.191956+0000", 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.191956+0000", 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.192013+0000", 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6983e-05 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.192035+0000", 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2132000000000001e-05 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.192061+0000", 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5323000000000001e-05 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.192305+0000", 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000244567 2024-09-21T23:11:06.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.192335+0000", 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0176e-05 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.193946+0000", 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016110569999999999 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.721 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(3.15 71/70 [PushOp(3:af6e50db:::rbd_data.11494fe4f7e7.0000000000000020:1a, version: 57'221, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:af6e50db:::rbd_data.11494fe4f7e7.0000000000000020:1a@57'221, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 1a=[]:{1a: [1a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:04.228358+0000", 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.4152495589999998, 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020311909999999999, 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.228358+0000", 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:06.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.228358+0000", 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.228359+0000", 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.16e-07 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.229081+0000", 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072214899999999995 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:06.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.229129+0000", 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8060999999999999e-05 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.229163+0000", 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4388999999999999e-05 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.229289+0000", 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125475 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.229316+0000", 2024-09-21T23:11:06.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6767e-05 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:04.230389+0000", 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010734340000000001 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:06.725 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:06.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:06.727 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:06.723+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:06.827 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:06.827+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:06.854 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:06.854 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:06.854 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:06.854 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:06.868 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:06.928 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:06.927+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:06.985 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:06.985 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:06.985 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:06.985 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:06.985 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:06.999 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:07.040 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:07.039+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:07.129 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:07.127+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:07.168 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:07.168 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:07.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:07.168 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:07.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.562767914999998, 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.562721247999999, 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:11:07.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:11:07.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:11:07.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.562647181999999, 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.562584456, 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:07.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:11:07.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.562532506, 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:11:07.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.562319420000001, 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:07.193 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.560435394999999, 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:07.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:07.197 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.560387344999999, 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.560339037999999, 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.560291823, 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:11:07.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.209 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.560227594000001, 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:11:07.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:07.213 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.560175684000001, 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:11:07.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.217 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40186 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 442368~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.555628081999998, 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093234290999999997, 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40186 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610103+0000", 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610104+0000", 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610109+0000", 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2140000000000004e-06 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610125+0000", 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5276000000000001e-05 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610139+0000", 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4477e-05 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703148+0000", 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093008570999999998 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703236+0000", 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7868000000000006e-05 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703337+0000", 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101742 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.555573389999999, 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:11:07.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.555518575000001, 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:11:07.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.555470850999999, 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.233 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.555403962, 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.234 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:11:07.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.551858683999999, 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:11:07.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:11:07.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.551811943000001, 2024-09-21T23:11:07.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:11:07.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:11:07.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.246 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.551765420999999, 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:11:07.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.251 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:07.252 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:07.253 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:07.251+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:07.354 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:07.351+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:07.385 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:07.385 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:07.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:07.455 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:07.451+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:07.555 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:07.555+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:07.566 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:07.567 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:07.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:07.656 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:07.655+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:07.705 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51023 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468768452999999, 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037458331999999997, 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51023 2024-09-21T23:11:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235893+0000", 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5100000000000001e-07 2024-09-21T23:11:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235895+0000", 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3240000000000001e-06 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235901+0000", 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0610000000000001e-06 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235906+0000", 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1830000000000002e-06 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272599+0000", 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036692726000000002 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272669+0000", 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9417000000000003e-05 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273351+0000", 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00068227000000000001 2024-09-21T23:11:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51024 3.4 3:2264aa58:::rbd_header.1149796cbcee:head [watch reconnect cookie 139807362363424 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468748592000001, 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039816437000000003, 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51024 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235913+0000", 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235915+0000", 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.897e-06 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235921+0000", 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3759999999999996e-06 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235926+0000", 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7759999999999997e-06 2024-09-21T23:11:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273380+0000", 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037453792999999999 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273460+0000", 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0241999999999995e-05 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275729+0000", 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022688740000000002 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51026 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468729121999999, 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038933657000000003, 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51026 2024-09-21T23:11:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.41e-07 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235935+0000", 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6369999999999999e-06 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235941+0000", 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.528e-06 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235947+0000", 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2959999999999998e-06 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274454+0000", 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038507844999999999 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274533+0000", 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.839e-05 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274865+0000", 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033262 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51028 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468708334999999, 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03952514, 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51028 2024-09-21T23:11:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235955+0000", 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0640000000000001e-06 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235961+0000", 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1099999999999999e-06 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235966+0000", 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4120000000000003e-06 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274896+0000", 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038930017999999997 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274973+0000", 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7616999999999999e-05 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275478+0000", 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000504544 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51030 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468689545, 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040194278, 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51030 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235972+0000", 2024-09-21T23:11:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235974+0000", 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3159999999999999e-06 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235980+0000", 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235985+0000", 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0359999999999997e-06 2024-09-21T23:11:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275761+0000", 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776211999999998 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275838+0000", 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7231999999999995e-05 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276166+0000", 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032743300000000002 2024-09-21T23:11:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51056 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468588195999999, 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040641749999999997, 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51056 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236080+0000", 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0180000000000004e-06 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236087+0000", 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8759999999999999e-06 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236092+0000", 2024-09-21T23:11:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276196+0000", 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040103449999999999 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276272+0000", 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6062000000000005e-05 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276715+0000", 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000442719 2024-09-21T23:11:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51057 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468561631, 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041665445000000002, 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51057 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236100+0000", 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236102+0000", 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0990000000000001e-06 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236110+0000", 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1610000000000003e-06 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236115+0000", 2024-09-21T23:11:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9880000000000004e-06 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276744+0000", 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040628876000000001 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276810+0000", 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6359999999999995e-05 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277765+0000", 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00095440200000000003 2024-09-21T23:11:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51058 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468539077999999, 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041431192999999998, 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51058 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236125+0000", 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236131+0000", 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4489999999999996e-06 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236136+0000", 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277067+0000", 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040930381000000002 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277212+0000", 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014488599999999999 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277553+0000", 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000341424 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51059 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.468518357000001, 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042250247999999997, 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51059 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236145+0000", 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236151+0000", 2024-09-21T23:11:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.268e-06 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236157+0000", 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4480000000000002e-06 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277579+0000", 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041421948 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277693+0000", 2024-09-21T23:11:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113786 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.278393+0000", 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00070025299999999996 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51324 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.451806478999998, 2024-09-21T23:11:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037384598999999998, 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51324 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252855+0000", 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252857+0000", 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2709999999999999e-06 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252864+0000", 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6889999999999998e-06 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252869+0000", 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1429999999999997e-06 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290081+0000", 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037211602000000003 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290154+0000", 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3200000000000004e-05 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290239+0000", 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5191000000000005e-05 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51325 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.451785049000001, 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037730817, 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51325 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252879+0000", 2024-09-21T23:11:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2239999999999998e-06 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252885+0000", 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3010000000000003e-06 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252890+0000", 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3569999999999997e-06 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290267+0000", 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037376615000000002 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290338+0000", 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1265000000000003e-05 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290607+0000", 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026868200000000001 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51338 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.451681138000001, 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038044604000000003, 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51338 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8099999999999998e-07 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252982+0000", 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3020000000000002e-06 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252989+0000", 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3149999999999997e-06 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252994+0000", 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9359999999999998e-06 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290639+0000", 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037645064999999998 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290713+0000", 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4115000000000004e-05 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291024+0000", 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031148999999999999 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51339 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.451659693, 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038794718999999998, 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51339 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253002+0000", 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253005+0000", 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253011+0000", 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4949999999999996e-06 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253016+0000", 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291053+0000", 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038037165999999997 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291126+0000", 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2575000000000002e-05 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291796+0000", 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067037100000000001 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51340 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.451625226000001, 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039295921999999997, 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51340 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9699999999999996e-07 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253039+0000", 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5220000000000002e-06 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253046+0000", 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253051+0000", 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1869999999999996e-06 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291828+0000", 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038776772000000001 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291914+0000", 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5847999999999995e-05 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292332+0000", 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000418136 2024-09-21T23:11:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51341 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.451603216999999, 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039551238000000002, 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51341 2024-09-21T23:11:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253064+0000", 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253072+0000", 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0339999999999997e-06 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253077+0000", 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.921e-06 2024-09-21T23:11:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292363+0000", 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039285863999999997 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292437+0000", 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4247999999999998e-05 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292609+0000", 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017169900000000001 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51475 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.448702215000001, 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037285393999999999, 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51475 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255960+0000", 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255965+0000", 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8760000000000004e-06 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255977+0000", 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2629000000000001e-05 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255985+0000", 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1179999999999992e-06 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292638+0000", 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036653036 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292713+0000", 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4262000000000002e-05 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293244+0000", 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00053145700000000004 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51477 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.448661156, 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037740821000000001, 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51477 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9599999999999999e-07 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256003+0000", 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.627e-06 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256011+0000", 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256016+0000", 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.784e-06 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293276+0000", 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037260003 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293363+0000", 2024-09-21T23:11:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6558999999999996e-05 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293741+0000", 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037767200000000001 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51479 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.448636520000001, 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038089583000000003, 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51479 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256025+0000", 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.13e-07 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256027+0000", 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1399999999999998e-06 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256034+0000", 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256038+0000", 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293776+0000", 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037737241999999997 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293852+0000", 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6287999999999999e-05 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294114+0000", 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026198699999999998 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51481 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.448615490000002, 2024-09-21T23:11:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038735549000000001, 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51481 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256046+0000", 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256048+0000", 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2359999999999999e-06 2024-09-21T23:11:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256055+0000", 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6669999999999999e-06 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256061+0000", 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6200000000000004e-06 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294146+0000", 2024-09-21T23:11:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038085073999999997 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294234+0000", 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8839999999999996e-05 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294781+0000", 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00054657999999999996 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.3369456900000003, 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:07.789 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:07.790 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:07.791 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:07.787+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:07.891 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:07.891+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:07.956 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:07.957 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:07.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:07.993 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:07.991+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:08.094 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:08.091+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:08.094 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:08.095 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:08.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:08.195 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:08.191+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.741933871000001, 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.741880432999999, 2024-09-21T23:11:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:11:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.741829458000002, 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:11:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.741778527000001, 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:11:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.741723216, 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:11:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.741654301000001, 2024-09-21T23:11:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:11:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.737016948000001, 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:11:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:11:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.736948807000001, 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:11:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:11:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:11:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.250795296, 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.249741920999998, 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.249695553999999, 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:11:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:11:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:11:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.249567375000002, 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:11:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.249481580000001, 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:11:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.248937765000001, 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:11:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.248839985, 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:11:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.248765111000001, 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.248704332999999, 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:11:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:11:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.248668015, 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:11:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.444206221000002, 2024-09-21T23:11:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:11:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:11:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:11:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9276680549999998, 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:08.321 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:08.322 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:08.323 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:08.323+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:08.402 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:11:08.448 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:08.447+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:08.472 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:08.472 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:08.474 INFO:teuthology.orchestra.run.smithi150.stdout:flattening image image_client.0-clone30 2024-09-21T23:11:08.501 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:08.523 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:08.523+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:08.624 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:08.623+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:08.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:08.664 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:08.664 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:08.664 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:08.664 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:08.664 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:08.683 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:08.725 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:08.723+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:08.726 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-21T23:11:08.742 INFO:teuthology.orchestra.run.smithi150.stdout:3654 read 0x452bdd8 thru 0x453a8cf (0xeaf8 bytes) 2024-09-21T23:11:08.746 INFO:teuthology.orchestra.run.smithi150.stdout:3656 trunc from 0xc1c30a9 to 0x9e7ab9c 2024-09-21T23:11:08.786 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:08.786 INFO:teuthology.orchestra.run.smithi150.stdout:3659 read 0x12a0574 thru 0x12aac60 (0xa6ed bytes) 2024-09-21T23:11:08.786 INFO:teuthology.orchestra.run.smithi150.stdout:3661 read 0xd297fd thru 0xd2b40d (0x1c11 bytes) 2024-09-21T23:11:08.786 INFO:teuthology.orchestra.run.smithi150.stdout:3665 read 0x33e684d thru 0x33ecc4b (0x63ff bytes) 2024-09-21T23:11:08.786 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:08.786 INFO:teuthology.orchestra.run.smithi150.stdout:3667 read 0x8ce4d38 thru 0x8ce93a7 (0x4670 bytes) 2024-09-21T23:11:08.797 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:08.797 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:08.799 INFO:teuthology.orchestra.run.smithi150.stdout:3668 write 0xe7dea31 thru 0xe7eaf3c (0xc50c bytes) 2024-09-21T23:11:08.809 INFO:teuthology.orchestra.run.smithi150.stdout:3669 read 0xc02896a thru 0xc0374a9 (0xeb40 bytes) 2024-09-21T23:11:08.817 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:08.817 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:08.823 INFO:teuthology.orchestra.run.smithi150.stdout:3671 trunc from 0xe7eaf3d to 0x3534e78 2024-09-21T23:11:08.826 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:08.823+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:08.839 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:08.839 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:08.861 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:08.861 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:08.867 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:08.867 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:08.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:08.885 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:08.886 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:08.903 INFO:teuthology.orchestra.run.smithi150.stdout:3672 writesame 0xcab999f thru 0xcac4079 data_size 0x1(0xa6db bytes) 2024-09-21T23:11:08.903 INFO:teuthology.orchestra.run.smithi150.stdout:3673 clone 32 order 20 su 65536 sc 10 2024-09-21T23:11:08.906 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:08.906 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:08.926 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:08.927 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:08.927 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:08.923+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:08.944 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:08.944 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:09.011 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:09.011 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:09.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:09.027 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:09.027+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:09.129 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:09.127+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:09.146 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:09.147 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:09.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:09.247 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:09.247+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:09.325 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:09.325 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:09.330 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:09.327+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:09.334 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:09.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:09.334 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-21T23:11:09.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-21T23:11:09.339 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:09.383 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:11:09.430 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:09.426+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:09.461 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone31 from 0xcac407a (overlap 0x3534e78) to 0x253ea20 2024-09-21T23:11:09.531 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:09.526+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:09.541 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:09.541 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:09.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:09.632 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-21T23:11:09.651 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:09.650+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:09.703 INFO:tasks.ceph.mon.a.smithi150.stderr:2024-09-21T23:11:09.699+0000 7f4d03887640 -1 mon.a@0(leader).osd e77 definitely_dead 0 2024-09-21T23:11:09.729 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:09.729 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:09.732 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:09.731+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:09.743 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:09.835 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:09.834+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:09.913 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:09.913 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:09.926 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:09.934 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:09.931+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:09.984 INFO:teuthology.orchestra.run.smithi150.stdout:3675 read 0x5435d60 thru 0x544027d (0xa51e bytes) 2024-09-21T23:11:09.985 INFO:teuthology.orchestra.run.smithi150.stdout:3678 read 0x9666221 thru 0x9669aaf (0x388f bytes) 2024-09-21T23:11:09.985 INFO:teuthology.orchestra.run.smithi150.stdout:3679 write 0x5257400 thru 0x52638ff (0xc500 bytes) 2024-09-21T23:11:10.001 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:10.001 INFO:teuthology.orchestra.run.smithi150.stdout:3682 writesame 0x8f3b4b9 thru 0x8f40ff6 data_size 0x33(0x5b3e bytes) 2024-09-21T23:11:10.008 INFO:teuthology.orchestra.run.smithi150.stdout:3684 read 0x9e65c4 thru 0x9f438b (0xddc8 bytes) 2024-09-21T23:11:10.009 INFO:teuthology.orchestra.run.smithi150.stdout:3685 read 0xbcc53b8 thru 0xbcca3eb (0x5034 bytes) 2024-09-21T23:11:10.009 INFO:teuthology.orchestra.run.smithi150.stdout:3686 write 0x3c43a04 thru 0x3c49ce0 (0x62dd bytes) 2024-09-21T23:11:10.017 INFO:teuthology.orchestra.run.smithi150.stdout:3688 trunc from 0xcac407a to 0xab5e44b 2024-09-21T23:11:10.034 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:10.030+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:10.045 INFO:teuthology.orchestra.run.smithi150.stdout:3689 write 0xc5ddca thru 0xc66867 (0x8a9e bytes) 2024-09-21T23:11:10.067 INFO:teuthology.orchestra.run.smithi150.stdout:3691 writesame 0xdbdefbb thru 0xdbeec4b data_size 0x29(0xfc91 bytes) 2024-09-21T23:11:10.072 INFO:teuthology.orchestra.run.smithi150.stdout:3692 trunc from 0xdbeec4c to 0xa5239ce 2024-09-21T23:11:10.094 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:10.094 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:10.109 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:10.115 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:10.115 INFO:teuthology.orchestra.run.smithi150.stdout:3695 write 0x5bcdbb6 thru 0x5bd0d8e (0x31d9 bytes) 2024-09-21T23:11:10.121 INFO:teuthology.orchestra.run.smithi150.stdout:3696 read 0x5c430b2 thru 0x5c50eec (0xde3b bytes) 2024-09-21T23:11:10.121 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:10.121 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:10.121 INFO:teuthology.orchestra.run.smithi150.stdout:3699 punch from 0x9e5584a to 0x9e57c40, (0x23f6 bytes) 2024-09-21T23:11:10.121 INFO:teuthology.orchestra.run.smithi150.stdout:3700 read 0x635042e thru 0x635c7a9 (0xc37c bytes) 2024-09-21T23:11:10.130 INFO:teuthology.orchestra.run.smithi150.stdout:3701 writesame 0xa5f3662 thru 0xa5f5b14 data_size 0x1(0x24b3 bytes) 2024-09-21T23:11:10.136 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:10.135+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:10.148 INFO:teuthology.orchestra.run.smithi150.stdout:3702 write 0xcfaf391 thru 0xcfb5ba8 (0x6818 bytes) 2024-09-21T23:11:10.157 INFO:teuthology.orchestra.run.smithi150.stdout:3703 trunc from 0xcfb5ba9 to 0xde56d7b 2024-09-21T23:11:10.182 INFO:teuthology.orchestra.run.smithi150.stdout:3704 write 0xe27e09c thru 0xe28ad26 (0xcc8b bytes) 2024-09-21T23:11:10.212 INFO:teuthology.orchestra.run.smithi150.stdout:3705 trunc from 0xe28ad27 to 0x37c5b67 2024-09-21T23:11:10.238 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:10.235+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:10.276 INFO:teuthology.orchestra.run.smithi150.stdout:3706 punch from 0xcfca00 to 0xcfddf7, (0x13f7 bytes) 2024-09-21T23:11:10.276 INFO:teuthology.orchestra.run.smithi150.stdout:3708 punch from 0x22cfc9d to 0x22d28dd, (0x2c40 bytes) 2024-09-21T23:11:10.276 INFO:teuthology.orchestra.run.smithi150.stdout:3709 punch from 0x2c67b62 to 0x2c728b2, (0xad50 bytes) 2024-09-21T23:11:10.276 INFO:teuthology.orchestra.run.smithi150.stdout:3710 read 0x15863e0 thru 0x158e129 (0x7d4a bytes) 2024-09-21T23:11:10.283 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:10.283 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:10.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:10.305 INFO:teuthology.orchestra.run.smithi150.stdout:3711 write 0xcae35d9 thru 0xcaef33e (0xbd66 bytes) 2024-09-21T23:11:10.311 INFO:teuthology.orchestra.run.smithi150.stdout:3712 read 0xc80af0d thru 0xc8162cb (0xb3bf bytes) 2024-09-21T23:11:10.311 INFO:teuthology.orchestra.run.smithi150.stdout:3713 read 0x5204733 thru 0x521100a (0xc8d8 bytes) 2024-09-21T23:11:10.311 INFO:teuthology.orchestra.run.smithi150.stdout:3714 writesame 0x65bd63 thru 0x65e5a8 data_size 0x5(0x2846 bytes) 2024-09-21T23:11:10.324 INFO:teuthology.orchestra.run.smithi150.stdout:3715 punch from 0x2ceee16 to 0x2cfc721, (0xd90b bytes) 2024-09-21T23:11:10.324 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:10.325 INFO:teuthology.orchestra.run.smithi150.stdout:3717 read 0x69e8317 thru 0x69f0877 (0x8561 bytes) 2024-09-21T23:11:10.325 INFO:teuthology.orchestra.run.smithi150.stdout:3718 writesame 0xc2cb541 thru 0xc2d24f0 data_size 0x8(0x6fb0 bytes) 2024-09-21T23:11:10.333 INFO:teuthology.orchestra.run.smithi150.stdout:3719 read 0x571a7d3 thru 0x571d81e (0x304c bytes) 2024-09-21T23:11:10.333 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:10.333 INFO:teuthology.orchestra.run.smithi150.stdout:3721 read 0x645a55 thru 0x64bfeb (0x6597 bytes) 2024-09-21T23:11:10.333 INFO:teuthology.orchestra.run.smithi150.stdout:3722 writesame 0xa01172f thru 0xa011d4c data_size 0x1b(0x61e bytes) 2024-09-21T23:11:10.337 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:10.335+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:10.339 INFO:teuthology.orchestra.run.smithi150.stdout:3723 punch from 0xae7aeb6 to 0xae8a824, (0xf96e bytes) 2024-09-21T23:11:10.339 INFO:teuthology.orchestra.run.smithi150.stdout:3724 read 0xba7197f thru 0xba7be63 (0xa4e5 bytes) 2024-09-21T23:11:10.354 INFO:teuthology.orchestra.run.smithi150.stdout:3725 writesame 0xd9d285d thru 0xd9d9228 data_size 0x7a(0x69cc bytes) 2024-09-21T23:11:10.354 INFO:teuthology.orchestra.run.smithi150.stdout:3726 writesame 0x7764a1f thru 0x776dbc2 data_size 0x4e(0x91a4 bytes) 2024-09-21T23:11:10.360 INFO:teuthology.orchestra.run.smithi150.stdout:3727 read 0x18de00f thru 0x18e6b2d (0x8b1f bytes) 2024-09-21T23:11:10.360 INFO:teuthology.orchestra.run.smithi150.stdout:3728 writesame 0x57dcb32 thru 0x57e1bdb data_size 0x46(0x50aa bytes) 2024-09-21T23:11:10.368 INFO:teuthology.orchestra.run.smithi150.stdout:3729 punch from 0x3184f76 to 0x31931de, (0xe268 bytes) 2024-09-21T23:11:10.368 INFO:teuthology.orchestra.run.smithi150.stdout:3730 punch from 0x8cf773e to 0x8cf8467, (0xd29 bytes) 2024-09-21T23:11:10.369 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:10.369 INFO:teuthology.orchestra.run.smithi150.stdout:3732 trunc from 0xd9d9229 to 0xd406aa7 2024-09-21T23:11:10.380 INFO:teuthology.orchestra.run.smithi150.stdout:3733 read 0x1e42aa9 thru 0x1e4fb4d (0xd0a5 bytes) 2024-09-21T23:11:10.380 INFO:teuthology.orchestra.run.smithi150.stdout:3734 read 0x5c23401 thru 0x5c2ff20 (0xcb20 bytes) 2024-09-21T23:11:10.396 INFO:teuthology.orchestra.run.smithi150.stderr:marked in osd.7. 2024-09-21T23:11:10.409 INFO:teuthology.orchestra.run.smithi150.stdout:3735 trunc from 0xd406aa7 to 0x1dbe872 2024-09-21T23:11:10.426 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-21T23:11:10.427 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:10.427 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:10.438 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:10.434+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:10.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:10.539 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:10.535+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:10.607 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:10.607 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:10.621 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:10.667 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:10.666+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:10.741 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:10.738+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:10.743 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:10.744 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:10.758 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:10.842 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:10.839+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:10.918 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:10.918 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:10.933 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:10.960 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:10.980 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:10.979+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:11.043 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:11.042+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:11.067 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:11.067 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:11.082 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:11.123 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:11.124 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:11.140 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:11.140 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:11.144 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:11.142+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:11.157 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:11.157 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:11.173 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:11.174 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:11.194 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:11.194 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:11.201 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:11.201 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:11.201 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:11.202 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:11.202 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:11.214 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:11.215 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:11.216 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:11.235 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:11.235 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:11.244 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:11.242+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:11.256 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:11.256 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:11.345 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:11.342+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:11.356 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.752965014000001, 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.752918347000001, 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:11.365 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.752844281000002, 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.369 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.752781554999999, 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:11:11.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:11:11.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.373 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.752729604999999, 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.752516519, 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:11:11.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:11:11.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:11:11.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:11:11.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.750632494000001, 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:11:11.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:11:11.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:11:11.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.750584444000001, 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:11:11.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.750536137000001, 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.750488921999999, 2024-09-21T23:11:11.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.750424692999999, 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:11.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:11:11.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.402 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.750372783, 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.403 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:11:11.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:11:11.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:11:11.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.745770489000002, 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:11:11.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.745715673999999, 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:11:11.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:11:11.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.745667950000001, 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.745601060999999, 2024-09-21T23:11:11.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.423 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.742055783000001, 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:11.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:11:11.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.742009041999999, 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.741962520000001, 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.9472663419999998, 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:11:11.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:11:11.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.440 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.441 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.441 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:11.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:11:11.442 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:11.447 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:11.442+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:11.547 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:11.543+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:11.607 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:11.608 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:11.608 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:11.623 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:11.648 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:11.646+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:11.748 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:11.747+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:11.784 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:11.784 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:11.784 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:11.784 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:11.784 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:11.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:11.888 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:11.887+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:11.950 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:11.950+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:11.968 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:11.968 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:11.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:11.968 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:11.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51023 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.73029039, 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037458331999999997, 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51023 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235893+0000", 2024-09-21T23:11:11.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5100000000000001e-07 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235895+0000", 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3240000000000001e-06 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235901+0000", 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0610000000000001e-06 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235906+0000", 2024-09-21T23:11:11.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1830000000000002e-06 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272599+0000", 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036692726000000002 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272669+0000", 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9417000000000003e-05 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273351+0000", 2024-09-21T23:11:11.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00068227000000000001 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51024 3.4 3:2264aa58:::rbd_header.1149796cbcee:head [watch reconnect cookie 139807362363424 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730270528999998, 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039816437000000003, 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51024 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235913+0000", 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235915+0000", 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.897e-06 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235921+0000", 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3759999999999996e-06 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235926+0000", 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7759999999999997e-06 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273380+0000", 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037453792999999999 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273460+0000", 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0241999999999995e-05 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275729+0000", 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022688740000000002 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51026 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730251059, 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038933657000000003, 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51026 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:11.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.41e-07 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235935+0000", 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6369999999999999e-06 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235941+0000", 2024-09-21T23:11:11.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.528e-06 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235947+0000", 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2959999999999998e-06 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274454+0000", 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038507844999999999 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274533+0000", 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.839e-05 2024-09-21T23:11:11.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274865+0000", 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033262 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51028 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730230272, 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03952514, 2024-09-21T23:11:11.981 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51028 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235955+0000", 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0640000000000001e-06 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235961+0000", 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1099999999999999e-06 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235966+0000", 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4120000000000003e-06 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274896+0000", 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038930017999999997 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274973+0000", 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7616999999999999e-05 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275478+0000", 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000504544 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51030 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:11.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730211482000001, 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040194278, 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51030 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235972+0000", 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235974+0000", 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3159999999999999e-06 2024-09-21T23:11:11.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235980+0000", 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235985+0000", 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0359999999999997e-06 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275761+0000", 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776211999999998 2024-09-21T23:11:11.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275838+0000", 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7231999999999995e-05 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276166+0000", 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032743300000000002 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51056 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730110133, 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040641749999999997, 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51056 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236080+0000", 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0180000000000004e-06 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236087+0000", 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8759999999999999e-06 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236092+0000", 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276196+0000", 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040103449999999999 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276272+0000", 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6062000000000005e-05 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276715+0000", 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000442719 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51057 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730083568000001, 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041665445000000002, 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51057 2024-09-21T23:11:11.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236100+0000", 2024-09-21T23:11:11.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236102+0000", 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0990000000000001e-06 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236110+0000", 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1610000000000003e-06 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236115+0000", 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9880000000000004e-06 2024-09-21T23:11:11.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276744+0000", 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040628876000000001 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276810+0000", 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6359999999999995e-05 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277765+0000", 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00095440200000000003 2024-09-21T23:11:11.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51058 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730061015, 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041431192999999998, 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:11.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51058 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:11.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236125+0000", 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236131+0000", 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4489999999999996e-06 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236136+0000", 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277067+0000", 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040930381000000002 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277212+0000", 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014488599999999999 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277553+0000", 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000341424 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51059 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.730040293999998, 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042250247999999997, 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51059 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236145+0000", 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236151+0000", 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.268e-06 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236157+0000", 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4480000000000002e-06 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277579+0000", 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041421948 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277693+0000", 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113786 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.278393+0000", 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00070025299999999996 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51324 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.713328416, 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037384598999999998, 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51324 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252855+0000", 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252857+0000", 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2709999999999999e-06 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252864+0000", 2024-09-21T23:11:12.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6889999999999998e-06 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252869+0000", 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1429999999999997e-06 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290081+0000", 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037211602000000003 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290154+0000", 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3200000000000004e-05 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290239+0000", 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5191000000000005e-05 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51325 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.713306985999999, 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037730817, 2024-09-21T23:11:12.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51325 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252879+0000", 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2239999999999998e-06 2024-09-21T23:11:12.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252885+0000", 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3010000000000003e-06 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252890+0000", 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3569999999999997e-06 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290267+0000", 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037376615000000002 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290338+0000", 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1265000000000003e-05 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290607+0000", 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026868200000000001 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.014 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51338 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.713203074999999, 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038044604000000003, 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51338 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8099999999999998e-07 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252982+0000", 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3020000000000002e-06 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252989+0000", 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3149999999999997e-06 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252994+0000", 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9359999999999998e-06 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290639+0000", 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037645064999999998 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290713+0000", 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4115000000000004e-05 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291024+0000", 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031148999999999999 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51339 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.713181630000001, 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038794718999999998, 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51339 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.019 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253002+0000", 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:12.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253005+0000", 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253011+0000", 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4949999999999996e-06 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253016+0000", 2024-09-21T23:11:12.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291053+0000", 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038037165999999997 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291126+0000", 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2575000000000002e-05 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291796+0000", 2024-09-21T23:11:12.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067037100000000001 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51340 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.713147162999999, 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039295921999999997, 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51340 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9699999999999996e-07 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253039+0000", 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5220000000000002e-06 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253046+0000", 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253051+0000", 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1869999999999996e-06 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291828+0000", 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038776772000000001 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291914+0000", 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5847999999999995e-05 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292332+0000", 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000418136 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51341 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.713125154, 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039551238000000002, 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.027 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51341 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:12.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253064+0000", 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253072+0000", 2024-09-21T23:11:12.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0339999999999997e-06 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253077+0000", 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.921e-06 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292363+0000", 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039285863999999997 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292437+0000", 2024-09-21T23:11:12.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4247999999999998e-05 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292609+0000", 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017169900000000001 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51475 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.710224151999999, 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037285393999999999, 2024-09-21T23:11:12.031 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51475 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255960+0000", 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255965+0000", 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8760000000000004e-06 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255977+0000", 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2629000000000001e-05 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255985+0000", 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1179999999999992e-06 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292638+0000", 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036653036 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292713+0000", 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4262000000000002e-05 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293244+0000", 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00053145700000000004 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51477 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:12.035 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.710183093000001, 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037740821000000001, 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51477 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9599999999999999e-07 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256003+0000", 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.627e-06 2024-09-21T23:11:12.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256011+0000", 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256016+0000", 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.784e-06 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293276+0000", 2024-09-21T23:11:12.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037260003 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293363+0000", 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6558999999999996e-05 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293741+0000", 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037767200000000001 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51479 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.710158456999999, 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038089583000000003, 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51479 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256025+0000", 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.13e-07 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256027+0000", 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1399999999999998e-06 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256034+0000", 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256038+0000", 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293776+0000", 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037737241999999997 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293852+0000", 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6287999999999999e-05 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294114+0000", 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026198699999999998 2024-09-21T23:11:12.043 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51481 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.710137426999999, 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038735549000000001, 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.044 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51481 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256046+0000", 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256048+0000", 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2359999999999999e-06 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256055+0000", 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6669999999999999e-06 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256061+0000", 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6200000000000004e-06 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294146+0000", 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038085073999999997 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294234+0000", 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8839999999999996e-05 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294781+0000", 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00054657999999999996 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.5984676269999998, 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.048 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:11:12.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:11:12.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:11:12.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.053 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:12.054 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:12.054 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:12.050+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:12.155 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:12.151+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:12.225 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:12.226 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:12.226 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:12.226 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:12.239 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:12.256 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:12.254+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:12.356 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:12.354+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:12.404 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:12.404 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:12.404 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:12.404 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:12.404 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:12.407 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:11:12.426 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:12.460 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:12.454+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:12.516 INFO:teuthology.orchestra.run.smithi150.stdout:3736 read 0x1575e24 thru 0x1582e3d (0xd01a bytes) 2024-09-21T23:11:12.517 INFO:teuthology.orchestra.run.smithi150.stdout:3737 write 0x11d0e77 thru 0x11d8144 (0x72ce bytes) 2024-09-21T23:11:12.521 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:12.541 INFO:teuthology.orchestra.run.smithi150.stdout:3740 writesame 0x88963c5 thru 0x88a0b5d data_size 0x1(0xa799 bytes) 2024-09-21T23:11:12.541 INFO:teuthology.orchestra.run.smithi150.stdout:3741 punch from 0x52be745 to 0x52ca589, (0xbe44 bytes) 2024-09-21T23:11:12.541 INFO:teuthology.orchestra.run.smithi150.stdout:3742 writesame 0x14e5abf thru 0x14e7e8e data_size 0x18(0x23d0 bytes) 2024-09-21T23:11:12.558 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:12.555+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.069321533, 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:11:12.562 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:11:12.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.069268095000002, 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.069217120000001, 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:11:12.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:11:12.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.069166189000001, 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.069110878, 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:12.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:11:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.069041963, 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:11:12.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:12.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.06440461, 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:11:12.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.064336469000001, 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:11:12.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.578182957999999, 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:11:12.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.577129583000001, 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.577083215999998, 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.576955037000001, 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.611 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:11:12.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.576869242000001, 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.616 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:11:12.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.620 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.576325427, 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:11:12.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:11:12.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.576227647, 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:11:12.625 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:11:12.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.629 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.576152773, 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.634 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.576091994999999, 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:12.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.576055676999999, 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:11:12.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.771593883000001, 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:11:12.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:11:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.649 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.255055716999999, 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:11:12.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:12.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:12.655 INFO:teuthology.orchestra.run.smithi150.stdout:3743 write 0xc6aeb13 thru 0xc6bb702 (0xcbf0 bytes) 2024-09-21T23:11:12.656 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:12.657 INFO:teuthology.orchestra.run.smithi150.stdout:3744 trunc from 0xc6bb703 to 0x2d03d75 2024-09-21T23:11:12.658 INFO:teuthology.orchestra.run.smithi150.stdout:3745 read 0x15ba219 thru 0x15c1f3d (0x7d25 bytes) 2024-09-21T23:11:12.658 INFO:teuthology.orchestra.run.smithi150.stdout:3746 writesame 0x401882d thru 0x401e211 data_size 0x3(0x59e5 bytes) 2024-09-21T23:11:12.658 INFO:teuthology.orchestra.run.smithi150.stdout:3747 read 0x3ebcda4 thru 0x3ec776d (0xa9ca bytes) 2024-09-21T23:11:12.658 INFO:teuthology.orchestra.run.smithi150.stdout:3748 write 0x7ae185a thru 0x7aedf5b (0xc702 bytes) 2024-09-21T23:11:12.658 INFO:teuthology.orchestra.run.smithi150.stdout:3749 write 0x2f49f36 thru 0x2f586a8 (0xe773 bytes) 2024-09-21T23:11:12.658 INFO:teuthology.orchestra.run.smithi150.stdout:3751 punch from 0x43ee522 to 0x43f9fbc, (0xba9a bytes) 2024-09-21T23:11:12.658 INFO:teuthology.orchestra.run.smithi150.stdout:3752 read 0x25601f0 thru 0x256fbe5 (0xf9f6 bytes) 2024-09-21T23:11:12.659 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:12.658+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:12.660 INFO:teuthology.orchestra.run.smithi150.stdout:3753 trunc from 0x7aedf5c to 0x17409c4 2024-09-21T23:11:12.691 INFO:teuthology.orchestra.run.smithi150.stdout:3754 read 0xa478b thru 0xaa3b8 (0x5c2e bytes) 2024-09-21T23:11:12.692 INFO:teuthology.orchestra.run.smithi150.stdout:3755 punch from 0x1440c38 to 0x1444c39, (0x4001 bytes) 2024-09-21T23:11:12.692 INFO:teuthology.orchestra.run.smithi150.stdout:3756 read 0x68df1b thru 0x698798 (0xa87e bytes) 2024-09-21T23:11:12.705 INFO:teuthology.orchestra.run.smithi150.stdout:3758 writesame 0x57b160b thru 0x57b8216 data_size 0x1e(0x6c0c bytes) 2024-09-21T23:11:12.705 INFO:teuthology.orchestra.run.smithi150.stdout:3759 punch from 0x4ec878b to 0x4ecbdaf, (0x3624 bytes) 2024-09-21T23:11:12.705 INFO:teuthology.orchestra.run.smithi150.stdout:3760 write 0x803d6 thru 0x8f8fe (0xf529 bytes) 2024-09-21T23:11:12.723 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:12.723 INFO:teuthology.orchestra.run.smithi150.stdout:3763 writesame 0x29c6889 thru 0x29ce694 data_size 0x6(0x7e0c bytes) 2024-09-21T23:11:12.746 INFO:teuthology.orchestra.run.smithi150.stdout:3764 write 0xc4b0a8e thru 0xc4bb39f (0xa912 bytes) 2024-09-21T23:11:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:3765 read 0x6ac5ffe thru 0x6acbec0 (0x5ec3 bytes) 2024-09-21T23:11:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:12.752 INFO:teuthology.orchestra.run.smithi150.stdout:3768 writesame 0x69461c8 thru 0x694e54d data_size 0x25(0x8386 bytes) 2024-09-21T23:11:12.760 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:12.759+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:12.767 INFO:teuthology.orchestra.run.smithi150.stdout:3770 write 0xe24b6fd thru 0xe25b548 (0xfe4c bytes) 2024-09-21T23:11:12.773 INFO:teuthology.orchestra.run.smithi150.stdout:3771 writesame 0xbab7c02 thru 0xbabff50 data_size 0x53(0x834f bytes) 2024-09-21T23:11:12.779 INFO:teuthology.orchestra.run.smithi150.stdout:3773 read 0xbed1b11 thru 0xbedee40 (0xd330 bytes) 2024-09-21T23:11:12.797 INFO:teuthology.orchestra.run.smithi150.stdout:3774 trunc from 0xe25b549 to 0x842f5b 2024-09-21T23:11:12.797 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:12.797 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "MOSDPGPushReply(3.15 80/79 [PushReplyOp(3:aab5b979:::rbd_data.1149f84cbfe6.0000000000000009:head)])", 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:12.797962+0000", 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00021842300000000001, 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024594800000000001, 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:12.797962+0000", 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:12.797962+0000", 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:12.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:12.797965+0000", 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.458e-06 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:12.797978+0000", 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3592e-05 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:12.798015+0000", 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6473999999999997e-05 2024-09-21T23:11:12.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:12.798032+0000", 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7223000000000001e-05 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:12.798136+0000", 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103631 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:12.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:12.802 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:11:12.802 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 1 2024-09-21T23:11:12.802 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:12.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:12.861 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:12.858+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:12.892 INFO:teuthology.orchestra.run.smithi150.stdout:3775 punch from 0x4ad13 to 0x55e19, (0xb106 bytes) 2024-09-21T23:11:12.910 INFO:teuthology.orchestra.run.smithi150.stdout:3776 write 0x91afbd3 thru 0x91b1190 (0x15be bytes) 2024-09-21T23:11:12.917 INFO:teuthology.orchestra.run.smithi150.stdout:3777 trunc from 0x91b1191 to 0x6cbf4d2 2024-09-21T23:11:12.932 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:12.932 INFO:teuthology.orchestra.run.smithi150.stdout:3780 read 0x4539026 thru 0x454643a (0xd415 bytes) 2024-09-21T23:11:12.934 INFO:teuthology.orchestra.run.smithi150.stdout:3781 trunc from 0x6cbf4d2 to 0x87af6f9 2024-09-21T23:11:12.942 INFO:teuthology.orchestra.run.smithi150.stdout:3782 write 0x4586b89 thru 0x4593764 (0xcbdc bytes) 2024-09-21T23:11:12.958 INFO:teuthology.orchestra.run.smithi150.stdout:3785 write 0xa3117f8 thru 0xa31c3ba (0xabc3 bytes) 2024-09-21T23:11:12.963 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:12.959+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:12.966 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:12.966 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:12.978 INFO:teuthology.orchestra.run.smithi150.stdout:3790 write 0xe170590 thru 0xe1739f8 (0x3469 bytes) 2024-09-21T23:11:12.982 INFO:teuthology.orchestra.run.smithi150.stdout:3791 writesame 0x7f88ed9 thru 0x7f8dd40 data_size 0x34(0x4e68 bytes) 2024-09-21T23:11:12.986 INFO:teuthology.orchestra.run.smithi150.stdout:3792 write 0xdca9b28 thru 0xdcb2438 (0x8911 bytes) 2024-09-21T23:11:12.989 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:12.989 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:12.989 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:12.989 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:12.989 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:12.991 INFO:teuthology.orchestra.run.smithi150.stdout:3793 read 0x9638466 thru 0x963ddc2 (0x595d bytes) 2024-09-21T23:11:12.992 INFO:teuthology.orchestra.run.smithi150.stdout:3794 writesame 0x836595c thru 0x836b511 data_size 0x3(0x5bb6 bytes) 2024-09-21T23:11:12.999 INFO:teuthology.orchestra.run.smithi150.stdout:3795 read 0x12de70c thru 0x12e379a (0x508f bytes) 2024-09-21T23:11:12.999 INFO:teuthology.orchestra.run.smithi150.stdout:3796 read 0x495ad2a thru 0x4964e22 (0xa0f9 bytes) 2024-09-21T23:11:13.000 INFO:teuthology.orchestra.run.smithi150.stdout:3798 write 0x8a5081b thru 0x8a5f17a (0xe960 bytes) 2024-09-21T23:11:13.004 INFO:teuthology.orchestra.run.smithi150.stdout:3799 write 0x5e6e10a thru 0x5e7c2dc (0xe1d3 bytes) 2024-09-21T23:11:13.007 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:13.011 INFO:teuthology.orchestra.run.smithi150.stdout:3800 read 0x348ebf7 thru 0x34910d5 (0x24df bytes) 2024-09-21T23:11:13.011 INFO:teuthology.orchestra.run.smithi150.stdout:3801 write 0x70f945a thru 0x7100afa (0x76a1 bytes) 2024-09-21T23:11:13.017 INFO:teuthology.orchestra.run.smithi150.stdout:3803 write 0x601667a thru 0x602402c (0xd9b3 bytes) 2024-09-21T23:11:13.023 INFO:teuthology.orchestra.run.smithi150.stdout:3804 writesame 0x7164db4 thru 0x7173534 data_size 0x2d(0xe781 bytes) 2024-09-21T23:11:13.028 INFO:teuthology.orchestra.run.smithi150.stdout:3805 write 0x146fec1 thru 0x14759b1 (0x5af1 bytes) 2024-09-21T23:11:13.033 INFO:teuthology.orchestra.run.smithi150.stdout:3806 punch from 0x67efb8d to 0x67fa2f9, (0xa76c bytes) 2024-09-21T23:11:13.033 INFO:teuthology.orchestra.run.smithi150.stdout:3807 read 0x41d3bca thru 0x41dd01a (0x9451 bytes) 2024-09-21T23:11:13.034 INFO:teuthology.orchestra.run.smithi150.stdout:3808 punch from 0x66c89e6 to 0x66c9577, (0xb91 bytes) 2024-09-21T23:11:13.034 INFO:teuthology.orchestra.run.smithi150.stdout:3809 read 0xd01fa20 thru 0xd02bd34 (0xc315 bytes) 2024-09-21T23:11:13.034 INFO:teuthology.orchestra.run.smithi150.stdout:3812 read 0x6be784a thru 0x6be8b09 (0x12c0 bytes) 2024-09-21T23:11:13.035 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:13.035 INFO:teuthology.orchestra.run.smithi150.stdout:3814 read 0x955f868 thru 0x956c075 (0xc80e bytes) 2024-09-21T23:11:13.041 INFO:teuthology.orchestra.run.smithi150.stdout:3815 trunc from 0xe1739f9 to 0x9e99011 2024-09-21T23:11:13.054 INFO:teuthology.orchestra.run.smithi150.stdout:3816 writesame 0x27faea4 thru 0x2807eb2 data_size 0x7(0xd00f bytes) 2024-09-21T23:11:13.063 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:13.059+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:13.072 INFO:teuthology.orchestra.run.smithi150.stdout:3817 write 0xc2efb40 thru 0xc2fcfd8 (0xd499 bytes) 2024-09-21T23:11:13.077 INFO:teuthology.orchestra.run.smithi150.stdout:3821 read 0x858bf18 thru 0x858ff57 (0x4040 bytes) 2024-09-21T23:11:13.077 INFO:teuthology.orchestra.run.smithi150.stdout:3822 punch from 0xc20dd87 to 0xc217e46, (0xa0bf bytes) 2024-09-21T23:11:13.086 INFO:teuthology.orchestra.run.smithi150.stdout:3823 write 0xd384c5e thru 0xd38f5d1 (0xa974 bytes) 2024-09-21T23:11:13.092 INFO:teuthology.orchestra.run.smithi150.stdout:3824 punch from 0x6eeae39 to 0x6eeb31f, (0x4e6 bytes) 2024-09-21T23:11:13.092 INFO:teuthology.orchestra.run.smithi150.stdout:3826 clone 33 order 18 su 16384 sc 6 2024-09-21T23:11:13.138 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.061979667000003, 2024-09-21T23:11:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:11:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.139 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:11:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.140 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:11:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:11:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:11:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:11:13.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:11:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:11:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:11:13.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:13.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:11:13.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:11:13.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:11:13.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:11:13.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 61.896212816000002, 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:11:13.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:11:13.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:11:13.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:38234 3.1e 3:7a5571f8:::rbd_data.1149ed0ffb75.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.168267661999998, 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043458748999999998, 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 38234 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968400+0000", 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6360000000000001e-06 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968406+0000", 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.81e-06 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968419+0000", 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3046e-05 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968426+0000", 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5440000000000004e-06 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968519+0000", 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3348000000000001e-05 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968668+0000", 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148264 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.977244+0000", 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085759600000000005 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011662+0000", 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034417917999999999 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011690+0000", 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8368999999999999e-05 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011780+0000", 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0450000000000003e-05 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011857+0000", 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6403999999999996e-05 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52394 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.778040301000001, 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039669796, 2024-09-21T23:11:13.160 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52394 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358626+0000", 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6800000000000001e-07 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358628+0000", 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358634+0000", 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358639+0000", 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.772e-06 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398288+0000", 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039648640999999998 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398295+0000", 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0720000000000002e-06 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52395 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.164 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.778019169, 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039678049999999999, 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52395 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:13.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358647+0000", 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358649+0000", 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569999999999998e-06 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358655+0000", 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.127e-06 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358660+0000", 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6820000000000002e-06 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398317+0000", 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039657215000000003 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398325+0000", 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4789999999999999e-06 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52396 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.168 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777999231999999, 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039688093000000001, 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52396 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358667+0000", 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358669+0000", 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1509999999999998e-06 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358676+0000", 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-06 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358681+0000", 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1420000000000001e-06 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398348+0000", 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039666202999999997 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398355+0000", 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.861e-06 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.172 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52397 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777973248999999, 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039702474000000001, 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52397 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358693+0000", 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-21T23:11:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358695+0000", 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358701+0000", 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358706+0000", 2024-09-21T23:11:13.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8550000000000001e-06 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398385+0000", 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039679157999999999 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398395+0000", 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9299999999999998e-06 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52398 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777951934000001, 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039712074999999999, 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52398 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:11:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358716+0000", 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0940000000000002e-06 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358722+0000", 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-06 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358727+0000", 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0760000000000002e-06 2024-09-21T23:11:13.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398418+0000", 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039691048 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398426+0000", 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52399 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777932022999998, 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039729353000000002, 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52399 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:11:13.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358736+0000", 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358743+0000", 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4119999999999998e-06 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358747+0000", 2024-09-21T23:11:13.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8749999999999999e-06 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398453+0000", 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039705424000000003 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398463+0000", 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0182e-05 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52432 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777912045000001, 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039740801999999999, 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52432 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358756+0000", 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358762+0000", 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358767+0000", 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6129999999999997e-06 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398487+0000", 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039720278999999997 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398494+0000", 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5619999999999998e-06 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52433 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:13.188 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777891444000002, 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039750774000000003, 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52433 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358775+0000", 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358777+0000", 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358783+0000", 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358788+0000", 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.135e-06 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398518+0000", 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039730389999999997 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398525+0000", 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52434 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:13.192 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777871675, 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039771847999999999, 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52434 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:13.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:11:13.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358796+0000", 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8899999999999999e-06 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358802+0000", 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7139999999999998e-06 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358807+0000", 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8029999999999996e-06 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398556+0000", 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039749022000000002 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398566+0000", 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0067e-05 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52435 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777852048, 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039820863999999997, 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52435 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8599999999999999e-07 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358816+0000", 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2180000000000002e-06 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358822+0000", 2024-09-21T23:11:13.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9939999999999997e-06 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358827+0000", 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7670000000000003e-06 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398624+0000", 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797341 2024-09-21T23:11:13.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398634+0000", 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0158e-05 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.201 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52499 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777563396000001, 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039762180000000001, 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52499 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359103+0000", 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359105+0000", 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359111+0000", 2024-09-21T23:11:13.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1120000000000001e-06 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359116+0000", 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398857+0000", 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039741132999999998 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398864+0000", 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5870000000000004e-06 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52500 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777543538, 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776995000000002, 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52500 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:13.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359123+0000", 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359124+0000", 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9750000000000001e-06 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359131+0000", 2024-09-21T23:11:13.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359135+0000", 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7520000000000004e-06 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398892+0000", 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039756315 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398899+0000", 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.43e-06 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52526 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.77752426, 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039791576000000002, 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.210 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52526 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:13.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359142+0000", 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359144+0000", 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.838e-06 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359150+0000", 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359154+0000", 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398925+0000", 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039770497000000002 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398933+0000", 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1319999999999994e-06 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52527 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777505017999999, 2024-09-21T23:11:13.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03980852, 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52527 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359163+0000", 2024-09-21T23:11:13.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.995e-06 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359169+0000", 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9710000000000001e-06 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359174+0000", 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0629999999999996e-06 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398961+0000", 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039786948000000003 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398969+0000", 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1580000000000005e-06 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52528 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.218 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777485417000001, 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039822428999999999, 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52528 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359181+0000", 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359187+0000", 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359192+0000", 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7509999999999997e-06 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359198+0000", 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9830000000000001e-06 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398995+0000", 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797036000000001 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399003+0000", 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52529 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.777460275999999, 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039831105999999998, 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.223 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52529 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359206+0000", 2024-09-21T23:11:13.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359208+0000", 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359214+0000", 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1e-06 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359219+0000", 2024-09-21T23:11:13.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7879999999999997e-06 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399029+0000", 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039810509000000001 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399036+0000", 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60091 3.1e 3:78318373:::rbd_data.114952cb5a61.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.303604095000001, 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.045016473000000001, 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60091 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833063+0000", 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249999999999999e-06 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833070+0000", 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7320000000000001e-06 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833086+0000", 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6776999999999999e-05 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833095+0000", 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5609999999999999e-06 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833185+0000", 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9736999999999998e-05 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833379+0000", 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019444699999999999 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.840335+0000", 2024-09-21T23:11:13.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069561240000000002 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877722+0000", 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037386662000000001 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877779+0000", 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7021999999999997e-05 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877939+0000", 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016059000000000001 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.878078+0000", 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138596 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:13.232 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:13.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:13.234 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:13.231+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:13.274 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:13.335 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:13.331+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:13.380 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:13.380 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:13.380 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:13.380 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:13.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:13.436 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:13.431+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:13.537 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:13.535+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:13.542 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:13.542 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:13.542 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:13.542 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:13.542 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:13.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:13.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-21T23:11:13.603+0000 7f18713af640 0 --2- 172.21.15.150:0/1107539189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c13bb90 0x7f186c1dbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T23:11:13.636 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:13.634+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:13.640 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:13.640 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:13.658 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:13.658 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:13.678 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:13.678 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:13.687 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:13.687 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86374877299999997, 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86367776500000004, 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:11:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:11:13.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:11:13.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86361347399999999, 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.696 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86358297500000003, 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.700 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:13.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86343498900000004, 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:11:13.704 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:13.708 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86337848399999995, 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.863323909, 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:11:13.712 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:11:13.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86326869100000003, 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.716 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:13.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:11:13.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:11:13.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86321406199999995, 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:11:13.720 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:13.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:11:13.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:11:13.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86309491199999999, 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.725 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86301747799999995, 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:13.733 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86295185600000002, 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:13.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:11:13.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:11:13.737 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86288752800000001, 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.738 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86282643999999997, 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.742 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:13.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:11:13.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:11:13.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86276597499999996, 2024-09-21T23:11:13.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.750 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86270177699999995, 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:11:13.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:11:13.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86262776900000004, 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.758 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86259034300000004, 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:11:13.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.763 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86254382100000004, 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:11:13.767 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.86248432600000002, 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:13.768 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:11:13.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:13.772 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:13.773 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:13.773 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:13.774 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:13.774 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:13.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:13.775 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:13.776 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:13.776 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:13.776 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:13.776 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:13.771+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:13.777 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:13.777 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:13.874 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:13.870+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:13.960 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:13.961 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:13.975 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:13.971+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:13.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:14.076 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:14.075+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:14.152 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:14.152 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:14.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:14.211 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:14.210+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:14.278 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:14.274+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:14.293 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:14.293 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:14.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:14.403 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:14.402+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:14.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:11:14.484 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:14.483+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:14.499 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:14.499 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:14.515 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:14.603 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:14.602+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:14.682 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:14.678+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:14.684 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:14.684 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:14.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:14.791 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:14.790+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:14.865 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:14.865 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:14.879 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:14.883 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:14.879+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:14.983 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:14.982+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:15.044 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:15.044 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:15.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:15.085 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:15.083+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:15.185 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:15.183+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:15.226 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:15.226 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:15.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:15.287 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:15.286+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:15.363 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:15.363 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:15.371 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:11:15.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:15.407 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:15.403+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:15.425 INFO:teuthology.orchestra.run.smithi150.stdout:leaving image image_client.0-clone32 intact 2024-09-21T23:11:15.427 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:15.427 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:15.427 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T23:11:15.427 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-21T23:11:15.487 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:15.486+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:15.514 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:15.514 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:15.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:15.588 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:15.587+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:15.607 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:15.608 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:15.608 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:15.608 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:15.608 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:15.610 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:15.610 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:15.611 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-21T23:11:15.655 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-21T23:11:15.666 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:15.667 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:15.689 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:15.686+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:15.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:15.790 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:15.786+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:15.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:15.814 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:15.814 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:15.814 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:15.815 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:15.817 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:15.817 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:15.817 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-21T23:11:15.856 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:15.857 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:15.873 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:15.891 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:15.890+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:15.933 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:15.933 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:15.950 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:15.950 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:15.968 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:15.968 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:15.986 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:15.986 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:15.992 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:15.991+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:16.007 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:16.007 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:16.007 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:16.007 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:16.007 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:16.007 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:16.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:16.009 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:16.009 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-21T23:11:16.027 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:16.027 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:16.032 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:16.032 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:16.032 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:16.032 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:16.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:16.048 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:16.048 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:16.069 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:16.069 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:16.093 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:16.090+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:16.160 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:16.161 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:16.161 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:16.161 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:16.161 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:16.169 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:16.169 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:16.169 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:16.169 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:16.172 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:16.173 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:16.173 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-21T23:11:16.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:16.195 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:16.190+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:16.296 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:16.291+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51023 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 0~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.086054310000002, 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037458331999999997, 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51023 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235892+0000", 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235893+0000", 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5100000000000001e-07 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235895+0000", 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3240000000000001e-06 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235901+0000", 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0610000000000001e-06 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235906+0000", 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1830000000000002e-06 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272599+0000", 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036692726000000002 2024-09-21T23:11:16.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.272669+0000", 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9417000000000003e-05 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273351+0000", 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00068227000000000001 2024-09-21T23:11:16.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51024 3.4 3:2264aa58:::rbd_header.1149796cbcee:head [watch reconnect cookie 139807362363424 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.086034449, 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039816437000000003, 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51024 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235912+0000", 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235913+0000", 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235915+0000", 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.897e-06 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235921+0000", 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3759999999999996e-06 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235926+0000", 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7759999999999997e-06 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273380+0000", 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037453792999999999 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.273460+0000", 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0241999999999995e-05 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275729+0000", 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022688740000000002 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51026 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:16.332 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.086014979000002, 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038933657000000003, 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51026 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235932+0000", 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.41e-07 2024-09-21T23:11:16.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235935+0000", 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6369999999999999e-06 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235941+0000", 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.528e-06 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235947+0000", 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2959999999999998e-06 2024-09-21T23:11:16.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274454+0000", 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038507844999999999 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274533+0000", 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.839e-05 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274865+0000", 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033262 2024-09-21T23:11:16.336 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51028 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.085994192000001, 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03952514, 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51028 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235953+0000", 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235955+0000", 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0640000000000001e-06 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235961+0000", 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1099999999999999e-06 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235966+0000", 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4120000000000003e-06 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274896+0000", 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038930017999999997 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.274973+0000", 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7616999999999999e-05 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275478+0000", 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000504544 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.341 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51030 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.085975401999999, 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040194278, 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51030 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235971+0000", 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235972+0000", 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235974+0000", 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3159999999999999e-06 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235980+0000", 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.235985+0000", 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0359999999999997e-06 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275761+0000", 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776211999999998 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.275838+0000", 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7231999999999995e-05 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276166+0000", 2024-09-21T23:11:16.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00032743300000000002 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51056 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.085874053000001, 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040641749999999997, 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51056 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236073+0000", 2024-09-21T23:11:16.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236080+0000", 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0180000000000004e-06 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236087+0000", 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8759999999999999e-06 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236092+0000", 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:16.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276196+0000", 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040103449999999999 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276272+0000", 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6062000000000005e-05 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276715+0000", 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000442719 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51057 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.085847487999999, 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041665445000000002, 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51057 2024-09-21T23:11:16.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236099+0000", 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236100+0000", 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236102+0000", 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0990000000000001e-06 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236110+0000", 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1610000000000003e-06 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236115+0000", 2024-09-21T23:11:16.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9880000000000004e-06 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276744+0000", 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040628876000000001 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.276810+0000", 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6359999999999995e-05 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277765+0000", 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00095440200000000003 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.353 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51058 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.085824935000002, 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041431192999999998, 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51058 2024-09-21T23:11:16.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236122+0000", 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236125+0000", 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236131+0000", 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4489999999999996e-06 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236136+0000", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277067+0000", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040930381000000002 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277212+0000", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014488599999999999 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277553+0000", 2024-09-21T23:11:16.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000341424 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51059 3.4 3:2131ec3b:::rbd_data.11494fe4f7e7.0000000000000010:1a [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.085804213999999, 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042250247999999997, 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51059 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236143+0000", 2024-09-21T23:11:16.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236145+0000", 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236151+0000", 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.268e-06 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.236157+0000", 2024-09-21T23:11:16.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4480000000000002e-06 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277579+0000", 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041421948 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.277693+0000", 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113786 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.278393+0000", 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00070025299999999996 2024-09-21T23:11:16.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51324 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.069092336000001, 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037384598999999998, 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51324 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252854+0000", 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252855+0000", 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252857+0000", 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2709999999999999e-06 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252864+0000", 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6889999999999998e-06 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252869+0000", 2024-09-21T23:11:16.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1429999999999997e-06 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290081+0000", 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037211602000000003 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290154+0000", 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3200000000000004e-05 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290239+0000", 2024-09-21T23:11:16.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5191000000000005e-05 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51325 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.069070906, 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037730817, 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51325 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252876+0000", 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252879+0000", 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2239999999999998e-06 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252885+0000", 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3010000000000003e-06 2024-09-21T23:11:16.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252890+0000", 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3569999999999997e-06 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290267+0000", 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037376615000000002 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290338+0000", 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1265000000000003e-05 2024-09-21T23:11:16.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290607+0000", 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026868200000000001 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51338 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.370 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.068966995, 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038044604000000003, 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51338 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252980+0000", 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8099999999999998e-07 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252982+0000", 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3020000000000002e-06 2024-09-21T23:11:16.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252989+0000", 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3149999999999997e-06 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.252994+0000", 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9359999999999998e-06 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290639+0000", 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037645064999999998 2024-09-21T23:11:16.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.290713+0000", 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4115000000000004e-05 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291024+0000", 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031148999999999999 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.374 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51339 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.068945549999999, 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038794718999999998, 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51339 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.375 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253001+0000", 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253002+0000", 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:16.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253005+0000", 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253011+0000", 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4949999999999996e-06 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253016+0000", 2024-09-21T23:11:16.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291053+0000", 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038037165999999997 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291126+0000", 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2575000000000002e-05 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291796+0000", 2024-09-21T23:11:16.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00067037100000000001 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51340 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.068911083, 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039295921999999997, 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51340 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253036+0000", 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9699999999999996e-07 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253039+0000", 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5220000000000002e-06 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253046+0000", 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253051+0000", 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1869999999999996e-06 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291828+0000", 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038776772000000001 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.291914+0000", 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5847999999999995e-05 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292332+0000", 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000418136 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51341 3.4 3:20b0f633:::rbd_data.11494fe4f7e7.0000000000000033:1a [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.068889074000001, 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039551238000000002, 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51341 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:16.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253058+0000", 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253064+0000", 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253072+0000", 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0339999999999997e-06 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.253077+0000", 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.921e-06 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292363+0000", 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039285863999999997 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292437+0000", 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4247999999999998e-05 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292609+0000", 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017169900000000001 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51475 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.065988072, 2024-09-21T23:11:16.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037285393999999999, 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51475 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255959+0000", 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255960+0000", 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255965+0000", 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8760000000000004e-06 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255977+0000", 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2629000000000001e-05 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.255985+0000", 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1179999999999992e-06 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292638+0000", 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.036653036 2024-09-21T23:11:16.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.292713+0000", 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4262000000000002e-05 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293244+0000", 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00053145700000000004 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.391 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51477 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.392 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:16.392 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.065947012999999, 2024-09-21T23:11:16.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037740821000000001, 2024-09-21T23:11:16.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51477 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256000+0000", 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9599999999999999e-07 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256003+0000", 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.627e-06 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256011+0000", 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:16.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256016+0000", 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.784e-06 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293276+0000", 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037260003 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293363+0000", 2024-09-21T23:11:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6558999999999996e-05 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293741+0000", 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037767200000000001 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51479 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:16.397 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.065922377, 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038089583000000003, 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51479 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256024+0000", 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256025+0000", 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.13e-07 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256027+0000", 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1399999999999998e-06 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256034+0000", 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256038+0000", 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293776+0000", 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037737241999999997 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.293852+0000", 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6287999999999999e-05 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294114+0000", 2024-09-21T23:11:16.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026198699999999998 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:51481 3.4 3:2295e1a5:::rbd_data.11494fe4f7e7.000000000000002f:1a [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.065901347, 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038735549000000001, 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51481 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256045+0000", 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256046+0000", 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256048+0000", 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2359999999999999e-06 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256055+0000", 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6669999999999999e-06 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.256061+0000", 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6200000000000004e-06 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294146+0000", 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.038085073999999997 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294234+0000", 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8839999999999996e-05 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.294781+0000", 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00054657999999999996 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.954231546999999, 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:11:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.411 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:16.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:16.413 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:16.414 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:16.415 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:16.415 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-21T23:11:16.415 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:16.410+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:16.451 INFO:teuthology.orchestra.run.smithi150.stdout:3827 punch from 0x4033e5f to 0x403e4bd, (0xa65e bytes) 2024-09-21T23:11:16.459 INFO:teuthology.orchestra.run.smithi150.stdout:3829 read 0xca72cd3 thru 0xca7af92 (0x82c0 bytes) 2024-09-21T23:11:16.460 INFO:teuthology.orchestra.run.smithi150.stdout:3832 read 0xa05d332 thru 0xa06ccac (0xf97b bytes) 2024-09-21T23:11:16.460 INFO:teuthology.orchestra.run.smithi150.stdout:3833 read 0x399dc22 thru 0x39a1025 (0x3404 bytes) 2024-09-21T23:11:16.460 INFO:teuthology.orchestra.run.smithi150.stdout:3834 writesame 0x19b2f43 thru 0x19bd547 data_size 0x1f(0xa605 bytes) 2024-09-21T23:11:16.474 INFO:teuthology.orchestra.run.smithi150.stdout:3835 read 0x6502a20 thru 0x650d845 (0xae26 bytes) 2024-09-21T23:11:16.474 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.488 INFO:teuthology.orchestra.run.smithi150.stdout:3844 write 0xe4246e2 thru 0xe42bac4 (0x73e3 bytes) 2024-09-21T23:11:16.496 INFO:teuthology.orchestra.run.smithi150.stdout:3846 write 0x6b14cb6 thru 0x6b1e654 (0x999f bytes) 2024-09-21T23:11:16.506 INFO:teuthology.orchestra.run.smithi150.stdout:truncating to largest ever: 0xee18f54 2024-09-21T23:11:16.507 INFO:teuthology.orchestra.run.smithi150.stdout:3847 trunc from 0xe42bac5 to 0xee18f54 2024-09-21T23:11:16.513 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:16.511+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:16.519 INFO:teuthology.orchestra.run.smithi150.stdout:3848 writesame 0x7a9d5a5 thru 0x7aa5e59 data_size 0x1(0x88b5 bytes) 2024-09-21T23:11:16.537 INFO:teuthology.orchestra.run.smithi150.stdout:3850 writesame 0xb26024b thru 0xb264261 data_size 0x3(0x4017 bytes) 2024-09-21T23:11:16.545 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.545 INFO:teuthology.orchestra.run.smithi150.stdout:3852 write 0x38499a5 thru 0x38578bc (0xdf18 bytes) 2024-09-21T23:11:16.556 INFO:teuthology.orchestra.run.smithi150.stdout:3853 write 0xac22be8 thru 0xac23316 (0x72f bytes) 2024-09-21T23:11:16.562 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.562 INFO:teuthology.orchestra.run.smithi150.stdout:3857 writesame 0xd6bdf54 thru 0xd6cbda5 data_size 0x8f(0xde52 bytes) 2024-09-21T23:11:16.587 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:16.587 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:16.588 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:16.588 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:16.588 INFO:teuthology.orchestra.run.smithi150.stdout:3859 trunc from 0xee18f54 to 0x4b7479f 2024-09-21T23:11:16.607 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:16.616 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:16.614+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:16.630 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:16.630 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:16.630 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:16.630 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:16.631 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:16.634 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:16.634 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:16.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-21T23:11:16.643 INFO:teuthology.orchestra.run.smithi150.stdout:3860 writesame 0xc2f8ee2 thru 0xc2fd86b data_size 0x1(0x498a bytes) 2024-09-21T23:11:16.643 INFO:teuthology.orchestra.run.smithi150.stdout:3863 read 0x1f62809 thru 0x1f6c6f8 (0x9ef0 bytes) 2024-09-21T23:11:16.649 INFO:teuthology.orchestra.run.smithi150.stdout:3865 trunc from 0xc2fd86c to 0x7291fa4 2024-09-21T23:11:16.659 INFO:teuthology.orchestra.run.smithi150.stdout:3866 punch from 0x6d6ff87 to 0x6d72c4b, (0x2cc4 bytes) 2024-09-21T23:11:16.678 INFO:teuthology.orchestra.run.smithi150.stdout:3867 write 0xd7625e0 thru 0xd764b54 (0x2575 bytes) 2024-09-21T23:11:16.684 INFO:teuthology.orchestra.run.smithi150.stdout:3868 read 0x96971f2 thru 0x96a3b03 (0xc912 bytes) 2024-09-21T23:11:16.687 INFO:teuthology.orchestra.run.smithi150.stdout:3869 trunc from 0xd764b55 to 0xb176518 2024-09-21T23:11:16.706 INFO:teuthology.orchestra.run.smithi150.stdout:3871 punch from 0x126520d to 0x12723a7, (0xd19a bytes) 2024-09-21T23:11:16.706 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.706 INFO:teuthology.orchestra.run.smithi150.stdout:3874 punch from 0x136f0f0 to 0x137023a, (0x114a bytes) 2024-09-21T23:11:16.706 INFO:teuthology.orchestra.run.smithi150.stdout:3875 writesame 0x755a320 thru 0x755b7a1 data_size 0xa(0x1482 bytes) 2024-09-21T23:11:16.710 INFO:teuthology.orchestra.run.smithi150.stdout:3876 read 0x6731ce thru 0x681a42 (0xe875 bytes) 2024-09-21T23:11:16.714 INFO:teuthology.orchestra.run.smithi150.stdout:3877 read 0x9338ad7 thru 0x933a9cf (0x1ef9 bytes) 2024-09-21T23:11:16.714 INFO:teuthology.orchestra.run.smithi150.stdout:3878 read 0x32e4ae thru 0x33c04d (0xdba0 bytes) 2024-09-21T23:11:16.716 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:16.715+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:16.716 INFO:teuthology.orchestra.run.smithi150.stdout:3879 write 0x38a8066 thru 0x38ae110 (0x60ab bytes) 2024-09-21T23:11:16.720 INFO:teuthology.orchestra.run.smithi150.stdout:3881 read 0xa125591 thru 0xa12e176 (0x8be6 bytes) 2024-09-21T23:11:16.720 INFO:teuthology.orchestra.run.smithi150.stdout:3882 read 0x6f7c430 thru 0x6f7c7b7 (0x388 bytes) 2024-09-21T23:11:16.721 INFO:teuthology.orchestra.run.smithi150.stdout:3883 read 0x938e312 thru 0x9390369 (0x2058 bytes) 2024-09-21T23:11:16.732 INFO:teuthology.orchestra.run.smithi150.stdout:3884 trunc from 0xb176518 to 0x18b4f7a 2024-09-21T23:11:16.757 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:16.757 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:16.757 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:16.757 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:16.757 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:16.767 INFO:teuthology.orchestra.run.smithi150.stdout:3885 read 0xb1a4db thru 0xb22936 (0x845c bytes) 2024-09-21T23:11:16.767 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.768 INFO:teuthology.orchestra.run.smithi150.stdout:3887 punch from 0x945310 to 0x94eb18, (0x9808 bytes) 2024-09-21T23:11:16.768 INFO:teuthology.orchestra.run.smithi150.stdout:3888 read 0x1359ea thru 0x144343 (0xe95a bytes) 2024-09-21T23:11:16.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:16.792 INFO:teuthology.orchestra.run.smithi150.stdout:3889 write 0xb468b74 thru 0xb470de3 (0x8270 bytes) 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:16.803 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:16.804 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:16.804 INFO:teuthology.orchestra.run.smithi150.stdout:3890 read 0xb0b1545 thru 0xb0b7088 (0x5b44 bytes) 2024-09-21T23:11:16.804 INFO:teuthology.orchestra.run.smithi150.stdout:3891 writesame 0x70f85d2 thru 0x70f9abd data_size 0x1a(0x14ec bytes) 2024-09-21T23:11:16.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:16.806 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:16.806 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:3892 read 0xfaf80f thru 0xfbe0d0 (0xe8c2 bytes) 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:3893 punch from 0x5b7b879 to 0x5b7e171, (0x28f8 bytes) 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:3895 read 0xac47ee1 thru 0xac480eb (0x20b bytes) 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:3899 punch from 0xbe279c to 0xbe458a, (0x1dee bytes) 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:3900 read 0x42398c9 thru 0x424876e (0xeea6 bytes) 2024-09-21T23:11:16.808 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:16.809 INFO:teuthology.orchestra.run.smithi150.stdout:3902 write 0x819372f thru 0x819faa9 (0xc37b bytes) 2024-09-21T23:11:16.829 INFO:teuthology.orchestra.run.smithi150.stdout:3903 trunc from 0xb470de4 to 0x9398df 2024-09-21T23:11:16.829 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:16.827+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:16.880 INFO:teuthology.orchestra.run.smithi150.stdout:3904 write 0x2668d8f thru 0x2675d8e (0xd000 bytes) 2024-09-21T23:11:16.897 INFO:teuthology.orchestra.run.smithi150.stdout:3905 trunc from 0x2675d8f to 0x88f10a1 2024-09-21T23:11:16.908 INFO:teuthology.orchestra.run.smithi150.stdout:3907 punch from 0x5e881c9 to 0x5e8e8a8, (0x66df bytes) 2024-09-21T23:11:16.908 INFO:teuthology.orchestra.run.smithi150.stdout:3908 write 0x888fca3 thru 0x889332c (0x368a bytes) 2024-09-21T23:11:16.918 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:16.915+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:16.923 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.430243257999997, 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:16.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:11:16.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.927 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.430189820000002, 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.928 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:16.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:11:16.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.430138845000002, 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:11:16.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.430087913999998, 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:11:16.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.430032603000001, 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:16.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:11:16.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:11:16.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.429963688000001, 2024-09-21T23:11:16.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:11:16.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.425326335000001, 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:11:16.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:11:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.425258194000001, 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.939104682999996, 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.938051307999999, 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:11:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.938004941000003, 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:11:16.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:11:16.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.937876762000002, 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:11:16.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:11:16.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.937790966999998, 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.937247151999998, 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.979 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.937149372, 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.937074498000001, 2024-09-21T23:11:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:11:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.937013720000003, 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:11:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.936977401999997, 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:11:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.132515607999999, 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:11:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:11:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:11:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:11:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 17.615977441999998, 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:11:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:11:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:17.012 INFO:teuthology.orchestra.run.smithi150.stdout:3910 write 0xce87bea thru 0xce8cf83 (0x539a bytes) 2024-09-21T23:11:17.013 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:17.014 INFO:teuthology.orchestra.run.smithi150.stdout:3911 write 0xe40dc04 thru 0xe412a4e (0x4e4b bytes) 2024-09-21T23:11:17.014 INFO:teuthology.orchestra.run.smithi150.stdout:3912 writesame 0xb028a8c thru 0xb02b3bb data_size 0x8(0x2930 bytes) 2024-09-21T23:11:17.014 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:17.014 INFO:teuthology.orchestra.run.smithi150.stdout:3916 writesame 0x2d811db thru 0x2d84ce2 data_size 0x4(0x3b08 bytes) 2024-09-21T23:11:17.015 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:11:17.015 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:11:17.015 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:11:17.015 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:11:17.015 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:11:17.016 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:11:17.016 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:11:17.017 INFO:teuthology.orchestra.run.smithi150.stdout:3917 trunc from 0xe412a4f to 0x44b6218 2024-09-21T23:11:17.017 INFO:teuthology.orchestra.run.smithi150.stdout:3918 trunc from 0x44b6218 to 0x36f6ed2 2024-09-21T23:11:17.017 INFO:teuthology.orchestra.run.smithi150.stdout:3919 read 0x2e5676f thru 0x2e5cacf (0x6361 bytes) 2024-09-21T23:11:17.017 INFO:teuthology.orchestra.run.smithi150.stdout:3920 write 0xc912f7 thru 0xc913ff (0x109 bytes) 2024-09-21T23:11:17.019 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:17.018+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3921 writesame 0xc3d6471 thru 0xc3e3d55 data_size 0x5(0xd8e5 bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3922 read 0xb5b1ba0 thru 0xb5bfc49 (0xe0aa bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3924 read 0xc099ff4 thru 0xc0a4fee (0xaffb bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3925 read 0xb9c9b50 thru 0xb9d9861 (0xfd12 bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3926 punch from 0x45551d7 to 0x455ec8c, (0x9ab5 bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3927 punch from 0x5fb8c57 to 0x5fc7f66, (0xf30f bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3928 read 0x6460bda thru 0x646c938 (0xbd5f bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3930 read 0x9b0ba0f thru 0x9b18ccb (0xd2bd bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3931 read 0xa009b9a thru 0xa013775 (0x9bdc bytes) 2024-09-21T23:11:17.034 INFO:teuthology.orchestra.run.smithi150.stdout:3932 read 0xa36fb9e thru 0xa37d680 (0xdae3 bytes) 2024-09-21T23:11:17.037 INFO:teuthology.orchestra.run.smithi150.stdout:3933 trunc from 0xc3e3d56 to 0x8f0b64a 2024-09-21T23:11:17.065 INFO:teuthology.orchestra.run.smithi150.stdout:3935 write 0xd9f5a83 thru 0xda0432a (0xe8a8 bytes) 2024-09-21T23:11:17.073 INFO:teuthology.orchestra.run.smithi150.stdout:3936 read 0x2718ce7 thru 0x27286d1 (0xf9eb bytes) 2024-09-21T23:11:17.089 INFO:teuthology.orchestra.run.smithi150.stdout:3937 trunc from 0xda0432b to 0xf14571 2024-09-21T23:11:17.121 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:17.118+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:17.134 INFO:teuthology.orchestra.run.smithi150.stdout:3938 write 0xda4f1bf thru 0xda5af45 (0xbd87 bytes) 2024-09-21T23:11:17.147 INFO:teuthology.orchestra.run.smithi150.stdout:3939 writesame 0x1f1f5b3 thru 0x1f2425d data_size 0x1(0x4cab bytes) 2024-09-21T23:11:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:17.156 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:17.156 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:17.156 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:17.164 INFO:teuthology.orchestra.run.smithi150.stdout:3940 write 0x511a3b thru 0x519420 (0x79e6 bytes) 2024-09-21T23:11:17.173 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:17.179 INFO:teuthology.orchestra.run.smithi150.stdout:3941 write 0x8742b2d thru 0x874992e (0x6e02 bytes) 2024-09-21T23:11:17.200 INFO:teuthology.orchestra.run.smithi150.stdout:3943 trunc from 0xda5af46 to 0x435b3f1 2024-09-21T23:11:17.220 INFO:teuthology.orchestra.run.smithi150.stdout:3945 read 0x962821 thru 0x96c8b8 (0xa098 bytes) 2024-09-21T23:11:17.221 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:17.218+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:17.222 INFO:teuthology.orchestra.run.smithi150.stdout:3946 trunc from 0x435b3f1 to 0x34158d2 2024-09-21T23:11:17.255 INFO:teuthology.orchestra.run.smithi150.stdout:3948 writesame 0xd11d052 thru 0xd12a477 data_size 0x5(0xd426 bytes) 2024-09-21T23:11:17.255 INFO:teuthology.orchestra.run.smithi150.stdout:3950 writesame 0x30d3caf thru 0x30d507a data_size 0xe(0x13cc bytes) 2024-09-21T23:11:17.260 INFO:teuthology.orchestra.run.smithi150.stdout:3951 read 0x30cc1db thru 0x30d5380 (0x91a6 bytes) 2024-09-21T23:11:17.264 INFO:teuthology.orchestra.run.smithi150.stdout:3952 write 0xac49062 thru 0xac4f9dd (0x697c bytes) 2024-09-21T23:11:17.271 INFO:teuthology.orchestra.run.smithi150.stdout:3953 write 0x1be1c8e thru 0x1bec1d9 (0xa54c bytes) 2024-09-21T23:11:17.279 INFO:teuthology.orchestra.run.smithi150.stdout:3954 write 0xbf7e1a4 thru 0xbf84661 (0x64be bytes) 2024-09-21T23:11:17.300 INFO:teuthology.orchestra.run.smithi150.stdout:3956 trunc from 0xd12a478 to 0x3f11ab7 2024-09-21T23:11:17.317 INFO:teuthology.orchestra.run.smithi150.stdout:3957 read 0x38d6e1 thru 0x38eb30 (0x1450 bytes) 2024-09-21T23:11:17.321 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:17.321 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:17.321 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:17.321 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:17.321 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:17.322 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:17.319+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:17.336 INFO:teuthology.orchestra.run.smithi150.stdout:3960 write 0x832ffd2 thru 0x8335310 (0x533f bytes) 2024-09-21T23:11:17.338 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:17.344 INFO:teuthology.orchestra.run.smithi150.stdout:3961 read 0x5a8b0ac thru 0x5a91be2 (0x6b37 bytes) 2024-09-21T23:11:17.344 INFO:teuthology.orchestra.run.smithi150.stdout:3962 write 0x26040c4 thru 0x26079ab (0x38e8 bytes) 2024-09-21T23:11:17.348 INFO:teuthology.orchestra.run.smithi150.stdout:3963 punch from 0x7d5948f to 0x7d5bc8c, (0x27fd bytes) 2024-09-21T23:11:17.348 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:17.349 INFO:teuthology.orchestra.run.smithi150.stdout:3965 clone 34 order 18 su 131072 sc 5 2024-09-21T23:11:17.407 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:11:17.423 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:17.418+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:17.461 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:17.461 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 66.386043376999993, 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:11:17.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:11:17.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 66.220276526000006, 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:17.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:11:17.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:11:17.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:38234 3.1e 3:7a5571f8:::rbd_data.1149ed0ffb75.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.492331372000002, 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043458748999999998, 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 38234 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968400+0000", 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6360000000000001e-06 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968406+0000", 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.81e-06 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968419+0000", 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3046e-05 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968426+0000", 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5440000000000004e-06 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968519+0000", 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3348000000000001e-05 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968668+0000", 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148264 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.977244+0000", 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085759600000000005 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011662+0000", 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034417917999999999 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011690+0000", 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8368999999999999e-05 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011780+0000", 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0450000000000003e-05 2024-09-21T23:11:17.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011857+0000", 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6403999999999996e-05 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52394 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.102104011000002, 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039669796, 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52394 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358626+0000", 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6800000000000001e-07 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358628+0000", 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358634+0000", 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358639+0000", 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.772e-06 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398288+0000", 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039648640999999998 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398295+0000", 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0720000000000002e-06 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52395 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.102082879000001, 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039678049999999999, 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52395 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358647+0000", 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358649+0000", 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569999999999998e-06 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358655+0000", 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.127e-06 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358660+0000", 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6820000000000002e-06 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398317+0000", 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039657215000000003 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398325+0000", 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4789999999999999e-06 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52396 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.102062942, 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039688093000000001, 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52396 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:17.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358667+0000", 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358669+0000", 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1509999999999998e-06 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358676+0000", 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-06 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358681+0000", 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1420000000000001e-06 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398348+0000", 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039666202999999997 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398355+0000", 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.861e-06 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52397 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.102036958999999, 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039702474000000001, 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52397 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358693+0000", 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358695+0000", 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358701+0000", 2024-09-21T23:11:17.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358706+0000", 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8550000000000001e-06 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398385+0000", 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039679157999999999 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398395+0000", 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9299999999999998e-06 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52398 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.102015644000002, 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039712074999999999, 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52398 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358716+0000", 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0940000000000002e-06 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358722+0000", 2024-09-21T23:11:17.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-06 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358727+0000", 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0760000000000002e-06 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398418+0000", 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039691048 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398426+0000", 2024-09-21T23:11:17.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52399 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101995732999999, 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039729353000000002, 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52399 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358736+0000", 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358743+0000", 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4119999999999998e-06 2024-09-21T23:11:17.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358747+0000", 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8749999999999999e-06 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398453+0000", 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039705424000000003 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398463+0000", 2024-09-21T23:11:17.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0182e-05 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52432 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101975755000002, 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039740801999999999, 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52432 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358756+0000", 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358762+0000", 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:11:17.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358767+0000", 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6129999999999997e-06 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398487+0000", 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039720278999999997 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398494+0000", 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5619999999999998e-06 2024-09-21T23:11:17.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52433 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101955153999999, 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039750774000000003, 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52433 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358775+0000", 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358777+0000", 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358783+0000", 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:11:17.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358788+0000", 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.135e-06 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398518+0000", 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039730389999999997 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398525+0000", 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:17.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52434 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101935385000001, 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039771847999999999, 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52434 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358796+0000", 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8899999999999999e-06 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358802+0000", 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7139999999999998e-06 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358807+0000", 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8029999999999996e-06 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398556+0000", 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039749022000000002 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398566+0000", 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0067e-05 2024-09-21T23:11:17.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52435 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101915758000001, 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039820863999999997, 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.516 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52435 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8599999999999999e-07 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358816+0000", 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2180000000000002e-06 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358822+0000", 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9939999999999997e-06 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358827+0000", 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7670000000000003e-06 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398624+0000", 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797341 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398634+0000", 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0158e-05 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52499 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101627105999999, 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039762180000000001, 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52499 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359103+0000", 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359105+0000", 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359111+0000", 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1120000000000001e-06 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359116+0000", 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398857+0000", 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039741132999999998 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398864+0000", 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5870000000000004e-06 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.523 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52500 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101607248000001, 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776995000000002, 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52500 2024-09-21T23:11:17.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359123+0000", 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359124+0000", 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9750000000000001e-06 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359131+0000", 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359135+0000", 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7520000000000004e-06 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398892+0000", 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039756315 2024-09-21T23:11:17.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398899+0000", 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.43e-06 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52526 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101587970000001, 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039791576000000002, 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52526 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359142+0000", 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359144+0000", 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.838e-06 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359150+0000", 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359154+0000", 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398925+0000", 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039770497000000002 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398933+0000", 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1319999999999994e-06 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52527 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101568728, 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03980852, 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.532 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52527 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359163+0000", 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.995e-06 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359169+0000", 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9710000000000001e-06 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359174+0000", 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0629999999999996e-06 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398961+0000", 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039786948000000003 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398969+0000", 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1580000000000005e-06 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52528 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101549126999998, 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039822428999999999, 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.536 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52528 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359181+0000", 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359187+0000", 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359192+0000", 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7509999999999997e-06 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359198+0000", 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9830000000000001e-06 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398995+0000", 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797036000000001 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399003+0000", 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52529 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.101523986, 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039831105999999998, 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52529 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:17.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359206+0000", 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359208+0000", 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359214+0000", 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1e-06 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359219+0000", 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7879999999999997e-06 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399029+0000", 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039810509000000001 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399036+0000", 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60091 3.1e 3:78318373:::rbd_data.114952cb5a61.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.627667805000002, 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.045016473000000001, 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:17.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60091 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:17.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833063+0000", 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249999999999999e-06 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833070+0000", 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7320000000000001e-06 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:17.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833086+0000", 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6776999999999999e-05 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833095+0000", 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5609999999999999e-06 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833185+0000", 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9736999999999998e-05 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833379+0000", 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019444699999999999 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.840335+0000", 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069561240000000002 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877722+0000", 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037386662000000001 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877779+0000", 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7021999999999997e-05 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877939+0000", 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016059000000000001 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.878078+0000", 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138596 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.549 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:17.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:17.550 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:17.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:17.552 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:17.547+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:17.590 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone33 from 0x8335311 (overlap 0x9398df) to 0x0 2024-09-21T23:11:17.652 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:17.650+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:17.715 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:17.715 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:17.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:17.753 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:17.751+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:17.760 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-21T23:11:17.854 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:17.851+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:17.899 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:17.899 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:17.899 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:17.899 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:17.899 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:17.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:17.955 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:17.951+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:18.040 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:18.040 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:18.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:18.040 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:18.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.214984179, 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:11:18.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2149131710000001, 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:11:18.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:11:18.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2148488799999999, 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:11:18.049 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:11:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2148183809999997, 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:18.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:11:18.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:11:18.057 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2146703949999997, 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:11:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2146138899999999, 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2145593149999998, 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:11:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:11:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2145040969999998, 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:11:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:11:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2144494679999998, 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:11:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:11:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.214330318, 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:11:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:11:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2142528840000004, 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:11:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2141872620000003, 2024-09-21T23:11:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:11:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2141229339999997, 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:11:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:11:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2140618459999999, 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:11:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2140013810000001, 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:11:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:11:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2139371829999996, 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:11:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2138631750000002, 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2138257489999997, 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:11:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:11:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2137792269999998, 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:11:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:11:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.2137197320000004, 2024-09-21T23:11:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:11:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:11:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:18.128 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:18.128 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:18.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:18.130 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:18.126+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:18.215 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:3967 write 0x8e0eb85 thru 0x8e1b562 (0xc9de bytes) 2024-09-21T23:11:18.229 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:18.226+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:18.234 INFO:teuthology.orchestra.run.smithi150.stdout:3968 write 0x459561 thru 0x4640ad (0xab4d bytes) 2024-09-21T23:11:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:3970 writesame 0x6eca0a3 thru 0x6ecafbc data_size 0x1(0xf1a bytes) 2024-09-21T23:11:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:3971 punch from 0x6e75306 to 0x6e83c6f, (0xe969 bytes) 2024-09-21T23:11:18.249 INFO:teuthology.orchestra.run.smithi150.stdout:3972 read 0x1dabd8e thru 0x1db6215 (0xa488 bytes) 2024-09-21T23:11:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:3973 punch from 0x8ce7bc2 to 0x8cf1168, (0x95a6 bytes) 2024-09-21T23:11:18.250 INFO:teuthology.orchestra.run.smithi150.stdout:3975 read 0x9415e1 thru 0x94eddb (0xd7fb bytes) 2024-09-21T23:11:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:3976 writesame 0x97e9548 thru 0x97eabd8 data_size 0x1(0x1691 bytes) 2024-09-21T23:11:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:3977 punch from 0x2cb2bf8 to 0x2cbcd88, (0xa190 bytes) 2024-09-21T23:11:18.259 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:3979 writesame 0xb989440 thru 0xb993fb8 data_size 0x1(0xab79 bytes) 2024-09-21T23:11:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:3982 read 0x363faa5 thru 0x3641749 (0x1ca5 bytes) 2024-09-21T23:11:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.270 INFO:teuthology.orchestra.run.smithi150.stdout:3984 writesame 0x785a7f1 thru 0x7861135 data_size 0xd(0x6945 bytes) 2024-09-21T23:11:18.276 INFO:teuthology.orchestra.run.smithi150.stdout:3985 write 0xa65cc66 thru 0xa660d84 (0x411f bytes) 2024-09-21T23:11:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:3987 punch from 0x8bf71a5 to 0x8bfafae, (0x3e09 bytes) 2024-09-21T23:11:18.281 INFO:teuthology.orchestra.run.smithi150.stdout:3989 write 0x9cd750e thru 0x9cd9c9f (0x2792 bytes) 2024-09-21T23:11:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:3990 write 0x6cdd660 thru 0x6ce944d (0xbdee bytes) 2024-09-21T23:11:18.289 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:18.289 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:3992 read 0x79d0d7b thru 0x79da29f (0x9525 bytes) 2024-09-21T23:11:18.293 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:3995 writesame 0xd0936ab thru 0xd09fbbb data_size 0x1(0xc511 bytes) 2024-09-21T23:11:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:3996 read 0x49ece4c thru 0x49ef912 (0x2ac7 bytes) 2024-09-21T23:11:18.302 INFO:teuthology.orchestra.run.smithi150.stdout:3997 writesame 0x8584a03 thru 0x858fb91 data_size 0x1(0xb18f bytes) 2024-09-21T23:11:18.305 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:18.305 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:18.305 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:18.305 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:18.308 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:18.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:4001 read 0xb966c50 thru 0xb96bd24 (0x50d5 bytes) 2024-09-21T23:11:18.328 INFO:teuthology.orchestra.run.smithi150.stdout:4002 writesame 0x3ecb53b thru 0x3ed672b data_size 0x1(0xb1f1 bytes) 2024-09-21T23:11:18.343 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:18.343 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:18.364 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:18.362+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:4003 write 0xad69431 thru 0xad76f3b (0xdb0b bytes) 2024-09-21T23:11:18.364 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:18.365 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:18.367 INFO:teuthology.orchestra.run.smithi150.stdout:4004 write 0xe94b56 thru 0xe9eac7 (0x9f72 bytes) 2024-09-21T23:11:18.371 INFO:teuthology.orchestra.run.smithi150.stdout:4007 writesame 0x4c96726 thru 0x4c9a3c9 data_size 0x2(0x3ca4 bytes) 2024-09-21T23:11:18.384 INFO:teuthology.orchestra.run.smithi150.stdout:4008 write 0xd481b9d thru 0xd482cb2 (0x1116 bytes) 2024-09-21T23:11:18.385 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:18.385 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:18.388 INFO:teuthology.orchestra.run.smithi150.stdout:4009 read 0x7c587c8 thru 0x7c5d4f4 (0x4d2d bytes) 2024-09-21T23:11:18.388 INFO:teuthology.orchestra.run.smithi150.stdout:4011 read 0xc0e95c7 thru 0xc0f68ca (0xd304 bytes) 2024-09-21T23:11:18.388 INFO:teuthology.orchestra.run.smithi150.stdout:4012 write 0x9fef3b8 thru 0x9ff53b7 (0x6000 bytes) 2024-09-21T23:11:18.399 INFO:teuthology.orchestra.run.smithi150.stdout:4014 writesame 0xe940c31 thru 0xe946236 data_size 0xe(0x5606 bytes) 2024-09-21T23:11:18.399 INFO:teuthology.orchestra.run.smithi150.stdout:4017 punch from 0x1dbcdd6 to 0x1dc6dab, (0x9fd5 bytes) 2024-09-21T23:11:18.399 INFO:teuthology.orchestra.run.smithi150.stdout:4020 write 0x22f32ec thru 0x22feb78 (0xb88d bytes) 2024-09-21T23:11:18.404 INFO:teuthology.orchestra.run.smithi150.stdout:4023 read 0x192c482 thru 0x193bb80 (0xf6ff bytes) 2024-09-21T23:11:18.404 INFO:teuthology.orchestra.run.smithi150.stdout:4024 write 0xc3309be thru 0xc3317db (0xe1e bytes) 2024-09-21T23:11:18.405 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:18.405 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:18.407 INFO:teuthology.orchestra.run.smithi150.stdout:4028 read 0xa4ce127 thru 0xa4d1e23 (0x3cfd bytes) 2024-09-21T23:11:18.407 INFO:teuthology.orchestra.run.smithi150.stdout:4029 write 0x9695d8 thru 0x9710fe (0x7b27 bytes) 2024-09-21T23:11:18.412 INFO:teuthology.orchestra.run.smithi150.stdout:4030 write 0x6404b18 thru 0x640f447 (0xa930 bytes) 2024-09-21T23:11:18.416 INFO:teuthology.orchestra.run.smithi150.stdout:4031 writesame 0xe0a4d88 thru 0xe0a6b8e data_size 0x1(0x1e07 bytes) 2024-09-21T23:11:18.424 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:18.425 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:18.425 INFO:teuthology.orchestra.run.smithi150.stdout:4032 trunc from 0xe946237 to 0xd44a15c 2024-09-21T23:11:18.431 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:18.427+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:18.437 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.438 INFO:teuthology.orchestra.run.smithi150.stdout:4034 write 0x7967190 thru 0x796f1e5 (0x8056 bytes) 2024-09-21T23:11:18.440 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.440 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.440 INFO:teuthology.orchestra.run.smithi150.stdout:4037 read 0x148518f thru 0x148ee7e (0x9cf0 bytes) 2024-09-21T23:11:18.440 INFO:teuthology.orchestra.run.smithi150.stdout:4038 punch from 0x8bc3f4b to 0x8bd1f57, (0xe00c bytes) 2024-09-21T23:11:18.441 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.441 INFO:teuthology.orchestra.run.smithi150.stdout:4040 write 0xbcb7f99 thru 0xbcbd10e (0x5176 bytes) 2024-09-21T23:11:18.444 INFO:teuthology.orchestra.run.smithi150.stdout:4042 punch from 0x77756ec to 0x7780c91, (0xb5a5 bytes) 2024-09-21T23:11:18.444 INFO:teuthology.orchestra.run.smithi150.stdout:4043 flatten 2024-09-21T23:11:18.463 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:11:18.509 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:18.509 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:18.513 INFO:teuthology.orchestra.run.smithi150.stdout:4044 read 0x5098e4 thru 0x5131f3 (0x9910 bytes) 2024-09-21T23:11:18.514 INFO:teuthology.orchestra.run.smithi150.stdout:4049 read 0xd17ec39 thru 0xd184d6b (0x6133 bytes) 2024-09-21T23:11:18.514 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.515 INFO:teuthology.orchestra.run.smithi150.stdout:4051 writesame 0x7f40e9b thru 0x7f4e923 data_size 0x43(0xda89 bytes) 2024-09-21T23:11:18.518 INFO:teuthology.orchestra.run.smithi150.stdout:4052 punch from 0x5dbda20 to 0x5dbfc2c, (0x220c bytes) 2024-09-21T23:11:18.518 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:18.524 INFO:teuthology.orchestra.run.smithi150.stdout:4054 write 0xd4f1e92 thru 0xd4f2328 (0x497 bytes) 2024-09-21T23:11:18.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:18.527 INFO:teuthology.orchestra.run.smithi150.stdout:4055 write 0x8a07edb thru 0x8a146b4 (0xc7da bytes) 2024-09-21T23:11:18.531 INFO:teuthology.orchestra.run.smithi150.stdout:4057 clone 35 order 20 su 65536 sc 13 2024-09-21T23:11:18.531 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:18.527+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:18.632 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:18.630+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:18.640 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:18.640 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:18.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:18.733 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:18.731+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:18.819 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:18.819 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:18.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:18.834 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:18.830+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:18.935 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:18.931+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:18.996 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:18.996 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:19.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:19.056 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:19.054+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:19.134 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:19.135 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:19.136 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:19.135+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:19.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:19.237 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:19.235+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:19.316 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:19.316 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:19.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:19.337 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:19.335+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:19.438 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:19.434+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:19.454 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:19.454 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:19.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:19.539 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:19.535+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:19.639 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:19.639 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:19.639 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:19.635+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:19.652 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:19.740 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:19.739+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:19.786 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:19.786 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:19.801 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:19.812 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-21T23:11:19.815 INFO:teuthology.orchestra.run.smithi150.stdout:4058 writesame 0x255d9fc thru 0x25601c7 data_size 0x12(0x27cc bytes) 2024-09-21T23:11:19.830 INFO:teuthology.orchestra.run.smithi150.stdout:4060 read 0xc903052 thru 0xc90527a (0x2229 bytes) 2024-09-21T23:11:19.830 INFO:teuthology.orchestra.run.smithi150.stdout:4061 read 0xc28717b thru 0xc296cb6 (0xfb3c bytes) 2024-09-21T23:11:19.830 INFO:teuthology.orchestra.run.smithi150.stdout:4062 write 0xc1fe52f thru 0xc20c1d8 (0xdcaa bytes) 2024-09-21T23:11:19.837 INFO:teuthology.orchestra.run.smithi150.stdout:4063 read 0x49f5c4d thru 0x49fc179 (0x652d bytes) 2024-09-21T23:11:19.837 INFO:teuthology.orchestra.run.smithi150.stdout:4064 write 0xce18083 thru 0xce1e05c (0x5fda bytes) 2024-09-21T23:11:19.841 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:19.839+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:19.843 INFO:teuthology.orchestra.run.smithi150.stdout:4065 read 0x45832e7 thru 0x458f96c (0xc686 bytes) 2024-09-21T23:11:19.846 INFO:teuthology.orchestra.run.smithi150.stdout:4067 trunc from 0xd4f2329 to 0xb54cc70 2024-09-21T23:11:19.889 INFO:teuthology.orchestra.run.smithi150.stdout:4069 punch from 0x1ec7545 to 0x1ecc54c, (0x5007 bytes) 2024-09-21T23:11:19.889 INFO:teuthology.orchestra.run.smithi150.stdout:4070 write 0x4a761b4 thru 0x4a82877 (0xc6c4 bytes) 2024-09-21T23:11:19.918 INFO:teuthology.orchestra.run.smithi150.stdout:4071 trunc from 0xb54cc70 to 0x1d4b094 2024-09-21T23:11:19.942 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:19.939+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:19.950 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:19.950 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:19.966 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:19.969 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:20.004 INFO:teuthology.orchestra.run.smithi150.stdout:4074 write 0xdbacab1 thru 0xdbbc9ad (0xfefd bytes) 2024-09-21T23:11:20.010 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:20.010 INFO:teuthology.orchestra.run.smithi150.stdout:4077 write 0x6b0a782 thru 0x6b0d294 (0x2b13 bytes) 2024-09-21T23:11:20.015 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:20.015 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:20.016 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:20.018 INFO:teuthology.orchestra.run.smithi150.stdout:4082 trunc from 0xdbbc9ae to 0xbf85d8f 2024-09-21T23:11:20.038 INFO:teuthology.orchestra.run.smithi150.stdout:4084 writesame 0x1bd0baa thru 0x1bdacba data_size 0x1(0xa111 bytes) 2024-09-21T23:11:20.042 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:20.038+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:20.096 INFO:teuthology.orchestra.run.smithi150.stdout:4085 trunc from 0xbf85d8f to 0xb5c4e39 2024-09-21T23:11:20.100 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:20.100 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:20.109 INFO:teuthology.orchestra.run.smithi150.stdout:4086 write 0x1fc55c thru 0x1fd406 (0xeab bytes) 2024-09-21T23:11:20.116 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:20.120 INFO:teuthology.orchestra.run.smithi150.stdout:4087 read 0x57489f3 thru 0x5751158 (0x8766 bytes) 2024-09-21T23:11:20.120 INFO:teuthology.orchestra.run.smithi150.stdout:4088 read 0x7c80a58 thru 0x7c8e99b (0xdf44 bytes) 2024-09-21T23:11:20.120 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:20.120 INFO:teuthology.orchestra.run.smithi150.stdout:4090 write 0x4f8da92 thru 0x4f8ec80 (0x11ef bytes) 2024-09-21T23:11:20.138 INFO:teuthology.orchestra.run.smithi150.stdout:4091 trunc from 0xb5c4e39 to 0x3bd0a7d 2024-09-21T23:11:20.143 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:20.139+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:20.176 INFO:teuthology.orchestra.run.smithi150.stdout:4094 write 0x8ee22b0 thru 0x8ee93fd (0x714e bytes) 2024-09-21T23:11:20.181 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:20.182 INFO:teuthology.orchestra.run.smithi150.stdout:4096 write 0x2e3a2c9 thru 0x2e3d94a (0x3682 bytes) 2024-09-21T23:11:20.188 INFO:teuthology.orchestra.run.smithi150.stdout:4097 writesame 0x14d14ae thru 0x14d72d5 data_size 0x2e(0x5e28 bytes) 2024-09-21T23:11:20.194 INFO:teuthology.orchestra.run.smithi150.stdout:4100 punch from 0x28c8f92 to 0x28d8048, (0xf0b6 bytes) 2024-09-21T23:11:20.215 INFO:teuthology.orchestra.run.smithi150.stdout:4101 writesame 0xe9edbfa thru 0xe9f0ec1 data_size 0x5(0x32c8 bytes) 2024-09-21T23:11:20.215 INFO:teuthology.orchestra.run.smithi150.stdout:4103 read 0x3bded01 thru 0x3be2187 (0x3487 bytes) 2024-09-21T23:11:20.215 INFO:teuthology.orchestra.run.smithi150.stdout:4104 read 0xa157c36 thru 0xa15aa2e (0x2df9 bytes) 2024-09-21T23:11:20.215 INFO:teuthology.orchestra.run.smithi150.stdout:4105 clone 36 order 23 su 524288 sc 7 2024-09-21T23:11:20.243 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:20.239+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:20.245 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:20.245 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:20.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:20.345 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:20.342+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:20.427 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:20.427 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:20.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:20.445 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:20.442+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:20.447 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:20.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:11:20.573 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:20.573 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:20.574 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:20.571+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:20.588 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone35 from 0xe9f0ec2 (overlap 0x1d4b094) to 0x2ede75 2024-09-21T23:11:20.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:20.666 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:20.666 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:20.666 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:20.663+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:20.685 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:20.685 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:20.703 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:20.703 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:20.723 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:20.723 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:20.729 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-21T23:11:20.745 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:20.745 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:20.748 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:20.746+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:20.765 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:20.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:20.765 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:20.766 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:20.767 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:20.767 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:20.784 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:20.788 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:20.788 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:20.809 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:20.810 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:20.849 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:20.846+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:20.919 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:20.919 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:20.919 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:20.919 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:20.919 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:20.934 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:20.980 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:20.979+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:21.051 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:21.047+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.564965946000001, 2024-09-21T23:11:21.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:21.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:11:21.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:11:21.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.564912507999999, 2024-09-21T23:11:21.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:11:21.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:21.069 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.564861532999998, 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:11:21.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:11:21.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:21.074 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.564810602000001, 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:11:21.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:21.079 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.564755290999997, 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:11:21.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:11:21.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.564686375999997, 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:21.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:11:21.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.560049022999998, 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:11:21.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.559980881999998, 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.095 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:11:21.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.073827371, 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:11:21.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:11:21.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.072773996000002, 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:11:21.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.072727628999999, 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.110 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:11:21.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.114 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.072599449999998, 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.115 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:11:21.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:11:21.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.072513655000002, 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.120 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:11:21.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:21.125 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.071969840000001, 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:11:21.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.130 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.071872059999997, 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.131 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:11:21.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.071797185999998, 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:21.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:11:21.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:11:21.140 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.071736408, 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.141 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:11:21.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:11:21.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.07170009, 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.146 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:11:21.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.267238296000002, 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:11:21.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 21.750700129999998, 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.157 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:11:21.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:11:21.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.162 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:21.164 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:21.164 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:21.162+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:21.169 INFO:teuthology.orchestra.run.smithi150.stdout:4106 read 0x626cf33 thru 0x626d4a0 (0x56e bytes) 2024-09-21T23:11:21.171 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:21.171 INFO:teuthology.orchestra.run.smithi150.stdout:4108 write 0x88eca8a thru 0x88fbda7 (0xf31e bytes) 2024-09-21T23:11:21.186 INFO:teuthology.orchestra.run.smithi150.stdout:4109 punch from 0xcc4cda0 to 0xcc50f94, (0x41f4 bytes) 2024-09-21T23:11:21.186 INFO:teuthology.orchestra.run.smithi150.stdout:4112 punch from 0xca38988 to 0xca41cdb, (0x9353 bytes) 2024-09-21T23:11:21.186 INFO:teuthology.orchestra.run.smithi150.stdout:4113 read 0x4f7ccfd thru 0x4f7f264 (0x2568 bytes) 2024-09-21T23:11:21.187 INFO:teuthology.orchestra.run.smithi150.stdout:4115 punch from 0x54b0f77 to 0x54b4b37, (0x3bc0 bytes) 2024-09-21T23:11:21.187 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:21.187 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:21.187 INFO:teuthology.orchestra.run.smithi150.stdout:4118 write 0x2b07581 thru 0x2b106a9 (0x9129 bytes) 2024-09-21T23:11:21.207 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:21.223 INFO:teuthology.orchestra.run.smithi150.stdout:4123 trunc from 0xe9f0ec2 to 0x1e54cc5 2024-09-21T23:11:21.265 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:21.263+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:21.303 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:21.303 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:21.303 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:21.303 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:21.317 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:21.320 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:21.338 INFO:teuthology.orchestra.run.smithi150.stdout:4125 writesame 0xbdd2b0d thru 0xbddefd0 data_size 0x1c(0xc4c4 bytes) 2024-09-21T23:11:21.338 INFO:teuthology.orchestra.run.smithi150.stdout:4126 write 0xada0bae thru 0xadac7a8 (0xbbfb bytes) 2024-09-21T23:11:21.344 INFO:teuthology.orchestra.run.smithi150.stdout:4127 punch from 0x649f998 to 0x64a696c, (0x6fd4 bytes) 2024-09-21T23:11:21.344 INFO:teuthology.orchestra.run.smithi150.stdout:4129 write 0x4b8f2da thru 0x4b9edb8 (0xfadf bytes) 2024-09-21T23:11:21.358 INFO:teuthology.orchestra.run.smithi150.stdout:4131 trunc from 0xbddefd1 to 0x5f4fead 2024-09-21T23:11:21.365 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:21.363+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:21.384 INFO:teuthology.orchestra.run.smithi150.stdout:4134 trunc from 0x5f4fead to 0xab971fb 2024-09-21T23:11:21.393 INFO:teuthology.orchestra.run.smithi150.stdout:4135 punch from 0x3d93945 to 0x3d9c617, (0x8cd2 bytes) 2024-09-21T23:11:21.403 INFO:teuthology.orchestra.run.smithi150.stdout:4136 trunc from 0xab971fb to 0x3074cc0 2024-09-21T23:11:21.428 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:21.428 INFO:teuthology.orchestra.run.smithi150.stdout:4138 read 0x1179028 thru 0x117b38d (0x2366 bytes) 2024-09-21T23:11:21.429 INFO:teuthology.orchestra.run.smithi150.stdout:4140 clone 37 order 18 su 1024 sc 11 2024-09-21T23:11:21.452 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:21.452 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:21.452 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:21.452 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:21.452 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:21.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:21.488 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:11:21.489 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:21.487+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:21.568 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:21.567+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:21.607 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone36 from 0x3074cc0 (overlap 0x1e54cc5) to 0x153b5bd 2024-09-21T23:11:21.609 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:21.609 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:21.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.533809622999996, 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:21.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:11:21.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:11:21.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11832 3.b 3:d135e6cb:::rbd_data.1149e13ad964.000000000000000e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.368042771999995, 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.040576098999999997, 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11832 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240453+0000", 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240455+0000", 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2170000000000001e-06 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240466+0000", 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0886999999999999e-05 2024-09-21T23:11:21.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240493+0000", 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240507+0000", 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4853e-05 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240589+0000", 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1969000000000001e-05 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.240739+0000", 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014989300000000001 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.245632+0000", 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048926109999999998 2024-09-21T23:11:21.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280711+0000", 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.035078638000000002 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280758+0000", 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7731000000000001e-05 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.280902+0000", 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014340499999999999 2024-09-21T23:11:21.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.281029+0000", 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127262 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:38234 3.1e 3:7a5571f8:::rbd_data.1149ed0ffb75.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.640097617999999, 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043458748999999998, 2024-09-21T23:11:21.622 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 38234 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968398+0000", 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968400+0000", 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6360000000000001e-06 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968406+0000", 2024-09-21T23:11:21.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.81e-06 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968419+0000", 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3046e-05 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968426+0000", 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5440000000000004e-06 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968519+0000", 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3348000000000001e-05 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.968668+0000", 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000148264 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:39.977244+0000", 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085759600000000005 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011662+0000", 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.034417917999999999 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011690+0000", 2024-09-21T23:11:21.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8368999999999999e-05 2024-09-21T23:11:21.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:21.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011780+0000", 2024-09-21T23:11:21.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0450000000000003e-05 2024-09-21T23:11:21.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:40.011857+0000", 2024-09-21T23:11:21.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6403999999999996e-05 2024-09-21T23:11:21.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.629 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52394 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249870256999998, 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039669796, 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.630 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52394 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358625+0000", 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358626+0000", 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6800000000000001e-07 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358628+0000", 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358634+0000", 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358639+0000", 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.772e-06 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398288+0000", 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039648640999999998 2024-09-21T23:11:21.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398295+0000", 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0720000000000002e-06 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52395 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.634 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249849124999997, 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039678049999999999, 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52395 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:21.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358646+0000", 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358647+0000", 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9000000000000002e-07 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358649+0000", 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569999999999998e-06 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358655+0000", 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.127e-06 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358660+0000", 2024-09-21T23:11:21.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6820000000000002e-06 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398317+0000", 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039657215000000003 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398325+0000", 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4789999999999999e-06 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.638 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52396 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249829188, 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039688093000000001, 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52396 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.639 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358666+0000", 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358667+0000", 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1600000000000001e-07 2024-09-21T23:11:21.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358669+0000", 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1509999999999998e-06 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358676+0000", 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-06 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358681+0000", 2024-09-21T23:11:21.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1420000000000001e-06 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398348+0000", 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039666202999999997 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398355+0000", 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.861e-06 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52397 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249803204999999, 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039702474000000001, 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52397 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.643 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358692+0000", 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358693+0000", 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-21T23:11:21.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358695+0000", 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.136e-06 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358701+0000", 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358706+0000", 2024-09-21T23:11:21.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8550000000000001e-06 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398385+0000", 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039679157999999999 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398395+0000", 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9299999999999998e-06 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52398 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249781890000001, 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039712074999999999, 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.647 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52398 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358714+0000", 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358716+0000", 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0940000000000002e-06 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358722+0000", 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-06 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358727+0000", 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0760000000000002e-06 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398418+0000", 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039691048 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398426+0000", 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.650 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52399 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249761978999999, 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039729353000000002, 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.651 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52399 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358734+0000", 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358736+0000", 2024-09-21T23:11:21.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:11:21.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358743+0000", 2024-09-21T23:11:21.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4119999999999998e-06 2024-09-21T23:11:21.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358747+0000", 2024-09-21T23:11:21.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8749999999999999e-06 2024-09-21T23:11:21.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398453+0000", 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039705424000000003 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398463+0000", 2024-09-21T23:11:21.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0182e-05 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52432 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249742001000001, 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039740801999999999, 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52432 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358754+0000", 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358756+0000", 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9700000000000002e-06 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358762+0000", 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358767+0000", 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6129999999999997e-06 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398487+0000", 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039720278999999997 2024-09-21T23:11:21.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398494+0000", 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5619999999999998e-06 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.663 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52433 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.663 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:21.663 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249721399999999, 2024-09-21T23:11:21.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039750774000000003, 2024-09-21T23:11:21.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.664 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.664 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.664 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.664 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.664 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52433 2024-09-21T23:11:21.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.664 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358774+0000", 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358775+0000", 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358777+0000", 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-21T23:11:21.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358783+0000", 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999997e-06 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358788+0000", 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.135e-06 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398518+0000", 2024-09-21T23:11:21.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039730389999999997 2024-09-21T23:11:21.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398525+0000", 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52434 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249701631000001, 2024-09-21T23:11:21.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039771847999999999, 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52434 2024-09-21T23:11:21.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358794+0000", 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358796+0000", 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8899999999999999e-06 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358802+0000", 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7139999999999998e-06 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358807+0000", 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8029999999999996e-06 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398556+0000", 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039749022000000002 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398566+0000", 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0067e-05 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52435 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249682004, 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039820863999999997, 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52435 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358814+0000", 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8599999999999999e-07 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358816+0000", 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2180000000000002e-06 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358822+0000", 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9939999999999997e-06 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.358827+0000", 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7670000000000003e-06 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398624+0000", 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797341 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398634+0000", 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0158e-05 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.677 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52499 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249393351999998, 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039762180000000001, 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52499 2024-09-21T23:11:21.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359102+0000", 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359103+0000", 2024-09-21T23:11:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359105+0000", 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359111+0000", 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1120000000000001e-06 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359116+0000", 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398857+0000", 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039741132999999998 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398864+0000", 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5870000000000004e-06 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52500 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249373493999997, 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039776995000000002, 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52500 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359122+0000", 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359123+0000", 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359124+0000", 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9750000000000001e-06 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359131+0000", 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359135+0000", 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7520000000000004e-06 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398892+0000", 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039756315 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398899+0000", 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.43e-06 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52526 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249354216, 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039791576000000002, 2024-09-21T23:11:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52526 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359141+0000", 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359142+0000", 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359144+0000", 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.838e-06 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359150+0000", 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9179999999999999e-06 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359154+0000", 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398925+0000", 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039770497000000002 2024-09-21T23:11:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398933+0000", 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1319999999999994e-06 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52527 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249334974, 2024-09-21T23:11:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.03980852, 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52527 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359161+0000", 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359163+0000", 2024-09-21T23:11:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.995e-06 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359169+0000", 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9710000000000001e-06 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359174+0000", 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0629999999999996e-06 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398961+0000", 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039786948000000003 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398969+0000", 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1580000000000005e-06 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52528 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249315373000002, 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039822428999999999, 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52528 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359180+0000", 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359181+0000", 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359187+0000", 2024-09-21T23:11:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359192+0000", 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7509999999999997e-06 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359198+0000", 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9830000000000001e-06 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.398995+0000", 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039797036000000001 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399003+0000", 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4019999999999996e-06 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:52529 3.15 3.b7136fd5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.249290232, 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039831105999999998, 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 52529 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359205+0000", 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359206+0000", 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:11:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359208+0000", 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0279999999999999e-06 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359214+0000", 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1e-06 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.359219+0000", 2024-09-21T23:11:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7879999999999997e-06 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399029+0000", 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.039810509000000001 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.399036+0000", 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60091 3.1e 3:78318373:::rbd_data.114952cb5a61.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.775434051000001, 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.045016473000000001, 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60091 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833062+0000", 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833063+0000", 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249999999999999e-06 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833070+0000", 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7320000000000001e-06 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833086+0000", 2024-09-21T23:11:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6776999999999999e-05 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833095+0000", 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5609999999999999e-06 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833185+0000", 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9736999999999998e-05 2024-09-21T23:11:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.833379+0000", 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019444699999999999 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.840335+0000", 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069561240000000002 2024-09-21T23:11:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877722+0000", 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.037386662000000001 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877779+0000", 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7021999999999997e-05 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.877939+0000", 2024-09-21T23:11:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016059000000000001 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:57.878078+0000", 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138596 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:21.709 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:21.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:21.711 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:21.707+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:21.779 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-21T23:11:21.794 INFO:teuthology.orchestra.run.smithi150.stdout:4141 read 0x1e5ae01 thru 0x1e696eb (0xe8eb bytes) 2024-09-21T23:11:21.811 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:21.810+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:21.817 INFO:teuthology.orchestra.run.smithi150.stdout:4142 writesame 0x2ec2da2 thru 0x2ed2955 data_size 0x59(0xfbb4 bytes) 2024-09-21T23:11:21.859 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:21.859 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:21.859 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:21.859 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:21.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:21.912 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:21.911+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:22.012 INFO:teuthology.orchestra.run.smithi150.stdout:4144 trunc from 0x3074cc0 to 0xe8c1664 2024-09-21T23:11:22.013 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:22.011+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:22.017 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:22.017 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:22.017 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:11:22.018 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:11:22.018 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:11:22.018 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:11:22.019 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:22.019 INFO:teuthology.orchestra.run.smithi150.stdout:4147 punch from 0xad3e72a to 0xad45eff, (0x77d5 bytes) 2024-09-21T23:11:22.019 INFO:teuthology.orchestra.run.smithi150.stdout:4148 write 0xd8ef27d thru 0xd8f4cd4 (0x5a58 bytes) 2024-09-21T23:11:22.054 INFO:teuthology.orchestra.run.smithi150.stdout:4150 trunc from 0xe8c1664 to 0x37672f6 2024-09-21T23:11:22.067 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:22.067 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:22.067 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:22.067 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:22.067 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:22.068 INFO:teuthology.orchestra.run.smithi150.stdout:4152 read 0xa0d960 thru 0xa17e57 (0xa4f8 bytes) 2024-09-21T23:11:22.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:22.085 INFO:teuthology.orchestra.run.smithi150.stdout:4153 writesame 0x450d090 thru 0x4514cf8 data_size 0x1(0x7c69 bytes) 2024-09-21T23:11:22.089 INFO:teuthology.orchestra.run.smithi150.stdout:4155 trunc from 0x4514cf9 to 0x1da5f33 2024-09-21T23:11:22.132 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:22.130+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:22.189 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:22.189 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:11:22.189 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:22.216 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:22.214+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:22.258 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4351687420000001, 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:11:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:11:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4350977339999993, 2024-09-21T23:11:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:11:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:11:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.435033443, 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:11:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:11:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:11:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:11:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:11:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4350029440000007, 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:11:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:11:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4348549580000007, 2024-09-21T23:11:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:11:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4347984529999991, 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:11:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:11:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4347438780000008, 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:11:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:11:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4346886600000008, 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:11:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:11:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4346340309999999, 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:22.300 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4345148810000001, 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:11:22.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4344374470000005, 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:11:22.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:11:22.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:11:22.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4343718249999995, 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:11:22.309 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:11:22.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.313 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4343074970000007, 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:11:22.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:11:22.317 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.434246409, 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.318 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:22.322 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4341859439999993, 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:22.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:11:22.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.326 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4341217460000006, 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.327 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:11:22.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:11:22.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:11:22.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4340477380000003, 2024-09-21T23:11:22.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:11:22.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:11:22.336 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4340103119999998, 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:11:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.43396379, 2024-09-21T23:11:22.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:11:22.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4339042949999996, 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:11:22.349 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.350 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.350 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.350 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:22.351 INFO:teuthology.orchestra.run.smithi150.stdout:4156 punch from 0x1add4c to 0x1b2a8f, (0x4d43 bytes) 2024-09-21T23:11:22.351 INFO:teuthology.orchestra.run.smithi150.stdout:4158 write 0xcf2373e thru 0xcf29e6f (0x6732 bytes) 2024-09-21T23:11:22.351 INFO:teuthology.orchestra.run.smithi150.stdout:4160 write 0x189ae1d thru 0x18a21c5 (0x73a9 bytes) 2024-09-21T23:11:22.352 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:22.347+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:22.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:22.452 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:22.450+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:22.479 INFO:teuthology.orchestra.run.smithi150.stdout:4161 write 0x1bdea48 thru 0x1becd8f (0xe348 bytes) 2024-09-21T23:11:22.485 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:22.485 INFO:teuthology.orchestra.run.smithi150.stdout:4164 read 0x81f8717 thru 0x82048b7 (0xc1a1 bytes) 2024-09-21T23:11:22.485 INFO:teuthology.orchestra.run.smithi150.stdout:4166 write 0x73e46cf thru 0x73ea28f (0x5bc1 bytes) 2024-09-21T23:11:22.498 INFO:teuthology.orchestra.run.smithi150.stdout:4167 write 0x48d7923 thru 0x48e5bc9 (0xe2a7 bytes) 2024-09-21T23:11:22.511 INFO:teuthology.orchestra.run.smithi150.stdout:4168 read 0x100378d thru 0x1006764 (0x2fd8 bytes) 2024-09-21T23:11:22.520 INFO:teuthology.orchestra.run.smithi150.stdout:4169 trunc from 0xcf29e70 to 0xa000b8b 2024-09-21T23:11:22.533 INFO:teuthology.orchestra.run.smithi150.stdout:4170 read 0x9649b4b thru 0x9653ce2 (0xa198 bytes) 2024-09-21T23:11:22.543 INFO:teuthology.orchestra.run.smithi150.stdout:4171 write 0xb539e95 thru 0xb545cdb (0xbe47 bytes) 2024-09-21T23:11:22.550 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:22.550 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:22.550 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:22.550 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:22.553 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:22.551+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:22.557 INFO:teuthology.orchestra.run.smithi150.stdout:4172 read 0xc4692c thru 0xc4a669 (0x3d3e bytes) 2024-09-21T23:11:22.560 INFO:teuthology.orchestra.run.smithi150.stdout:4174 read 0x59d496b thru 0x59d8abb (0x4151 bytes) 2024-09-21T23:11:22.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:22.570 INFO:teuthology.orchestra.run.smithi150.stdout:4176 write 0xd811782 thru 0xd81d3ac (0xbc2b bytes) 2024-09-21T23:11:22.609 INFO:teuthology.orchestra.run.smithi150.stdout:4177 trunc from 0xd81d3ad to 0x1ff59b 2024-09-21T23:11:22.655 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:22.654+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:22.755 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:22.754+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:22.759 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:22.759 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:22.759 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:22.759 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:22.759 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:22.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:22.832 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:22.860 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:22.858+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:22.948 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:22.948 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:22.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:22.948 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:22.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.948 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.219978531000002, 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.949 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:11:22.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:11:22.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:22.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57266503800000002, 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:11:22.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:11:22.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.961 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57263236299999998, 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:11:22.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:11:22.965 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57260902800000002, 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.966 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:11:22.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.970 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57257646299999998, 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.971 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57248884099999997, 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.976 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:11:22.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:11:22.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.980 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57237009400000005, 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:11:22.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:11:22.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57234532900000001, 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:22.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:22.989 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57232418200000001, 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:22.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:11:22.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57226648499999999, 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:11:22.994 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.998 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57223160399999995, 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:22.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57220607899999998, 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:11:23.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:11:23.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57213489299999998, 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:11:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.012 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57209904700000003, 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:11:23.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57208195399999995, 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.017 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.021 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57206594300000002, 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:11:23.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57200869499999996, 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.026 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:23.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.030 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57192732999999996, 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:11:23.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57189577999999996, 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.035 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:11:23.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:11:23.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.57174888300000004, 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:23.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:11:23.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:11:23.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:23.044 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:23.045 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:23.045 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:23.046 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:23.046 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:23.047 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:23.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:23.048 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:23.042+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:23.048 INFO:teuthology.orchestra.run.smithi150.stdout:4178 write 0xdbb6b69 thru 0xdbc505d (0xe4f5 bytes) 2024-09-21T23:11:23.048 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:23.048 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:23.049 INFO:teuthology.orchestra.run.smithi150.stdout:4179 read 0x6b389e4 thru 0x6b44bdf (0xc1fc bytes) 2024-09-21T23:11:23.049 INFO:teuthology.orchestra.run.smithi150.stdout:4181 read 0x67a554e thru 0x67b3e85 (0xe938 bytes) 2024-09-21T23:11:23.049 INFO:teuthology.orchestra.run.smithi150.stdout:4182 read 0x11d939e thru 0x11e902c (0xfc8f bytes) 2024-09-21T23:11:23.049 INFO:teuthology.orchestra.run.smithi150.stdout:4184 read 0xb431877 thru 0xb435df7 (0x4581 bytes) 2024-09-21T23:11:23.049 INFO:teuthology.orchestra.run.smithi150.stdout:4186 trunc from 0xdbc505e to 0x6d0ef25 2024-09-21T23:11:23.050 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:23.050 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:23.059 INFO:teuthology.orchestra.run.smithi150.stdout:4187 clone 38 order 24 su 1048576 sc 10 2024-09-21T23:11:23.064 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:23.064 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:23.083 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:23.083 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:23.102 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:23.102 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:23.120 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:23.120 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:23.146 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:23.142+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:23.213 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:23.214 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:23.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:23.247 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:23.243+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:23.348 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:23.346+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:23.396 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:23.396 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:23.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:23.449 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:23.447+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:23.515 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:11:23.560 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:23.559+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:23.580 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:23.580 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:23.596 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:23.623 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone37 from 0x6d0ef25 (overlap 0x1ff59b) to 0x0 2024-09-21T23:11:23.650 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:23.647+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:23.723 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-21T23:11:23.725 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:23.725 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:23.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:23.751 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:23.747+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:23.766 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:23.772 INFO:teuthology.orchestra.run.smithi150.stdout:4191 trunc from 0x6d0ef25 to 0x26e83d4 2024-09-21T23:11:23.824 INFO:teuthology.orchestra.run.smithi150.stdout:4192 read 0x25a107b thru 0x25a626b (0x51f1 bytes) 2024-09-21T23:11:23.829 INFO:teuthology.orchestra.run.smithi150.stdout:4193 trunc from 0x26e83d4 to 0x58e5621 2024-09-21T23:11:23.840 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:23.843 INFO:teuthology.orchestra.run.smithi150.stdout:4196 trunc from 0x58e5621 to 0x76326c6 2024-09-21T23:11:23.852 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:23.850+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:23.853 INFO:teuthology.orchestra.run.smithi150.stdout:4198 clone 39 order 22 su 524288 sc 2 2024-09-21T23:11:23.871 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:23.871 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:23.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:23.953 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:23.951+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:24.054 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:24.054 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:24.054 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:24.050+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:24.068 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:24.155 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:24.154+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:24.202 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:24.202 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:24.216 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:24.260 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:24.259+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:24.339 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:24.340 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:24.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:24.356 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:24.354+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:24.457 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:24.455+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:11:24.525 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:24.525 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:24.539 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:24.558 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:24.554+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:24.658 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:24.655+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:24.700 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone38 from 0x76326c6 (overlap 0x26e83d4) to 0x0 2024-09-21T23:11:24.712 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:24.712 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:24.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:24.768 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:24.767+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:24.810 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-21T23:11:24.816 INFO:teuthology.orchestra.run.smithi150.stdout:4200 write 0x4e4e67f thru 0x4e56156 (0x7ad8 bytes) 2024-09-21T23:11:24.829 INFO:teuthology.orchestra.run.smithi150.stdout:4201 punch from 0x2b0525 to 0x2b2c02, (0x26dd bytes) 2024-09-21T23:11:24.834 INFO:teuthology.orchestra.run.smithi150.stdout:4202 trunc from 0x76326c6 to 0x4d9cf7d 2024-09-21T23:11:24.860 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:24.859+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:24.881 INFO:teuthology.orchestra.run.smithi150.stdout:4203 write 0xea07938 thru 0xea0ab11 (0x31da bytes) 2024-09-21T23:11:24.888 INFO:teuthology.orchestra.run.smithi150.stdout:4204 read 0xd5f4e17 thru 0xd5ff971 (0xab5b bytes) 2024-09-21T23:11:24.888 INFO:teuthology.orchestra.run.smithi150.stdout:4205 read 0x2517032 thru 0x252676c (0xf73b bytes) 2024-09-21T23:11:24.889 INFO:teuthology.orchestra.run.smithi150.stdout:4206 punch from 0x4a58108 to 0x4a594ae, (0x13a6 bytes) 2024-09-21T23:11:24.889 INFO:teuthology.orchestra.run.smithi150.stdout:4207 read 0x870e76e thru 0x8712f7a (0x480d bytes) 2024-09-21T23:11:24.889 INFO:teuthology.orchestra.run.smithi150.stdout:4208 write 0xdbca37 thru 0xdc4219 (0x77e3 bytes) 2024-09-21T23:11:24.895 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:24.895 INFO:teuthology.orchestra.run.smithi150.stdout:4211 read 0xe811490 thru 0xe81f83a (0xe3ab bytes) 2024-09-21T23:11:24.901 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:24.901 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:24.910 INFO:teuthology.orchestra.run.smithi150.stdout:4212 trunc from 0xea0ab12 to 0x5c644f2 2024-09-21T23:11:24.923 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:24.930 INFO:teuthology.orchestra.run.smithi150.stdout:4213 read 0x5b10858 thru 0x5b133bb (0x2b64 bytes) 2024-09-21T23:11:24.946 INFO:teuthology.orchestra.run.smithi150.stdout:4214 write 0x8076a55 thru 0x80820b2 (0xb65e bytes) 2024-09-21T23:11:24.953 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:24.954 INFO:teuthology.orchestra.run.smithi150.stdout:4217 write 0x24b4486 thru 0x24b4f5f (0xada bytes) 2024-09-21T23:11:24.958 INFO:teuthology.orchestra.run.smithi150.stdout:4218 read 0x1a9b9cb thru 0x1aa860d (0xcc43 bytes) 2024-09-21T23:11:24.959 INFO:teuthology.orchestra.run.smithi150.stdout:4219 read 0x57d3b52 thru 0x57d4cb5 (0x1164 bytes) 2024-09-21T23:11:24.959 INFO:teuthology.orchestra.run.smithi150.stdout:4222 writesame 0x34fcafb thru 0x34ffec9 data_size 0x1(0x33cf bytes) 2024-09-21T23:11:24.961 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:24.958+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:25.004 INFO:teuthology.orchestra.run.smithi150.stdout:4226 writesame 0xed840f7 thru 0xed84f1b data_size 0x3(0xe25 bytes) 2024-09-21T23:11:25.004 INFO:teuthology.orchestra.run.smithi150.stdout:4230 read 0x2acd9cd thru 0x2ad933c (0xb970 bytes) 2024-09-21T23:11:25.005 INFO:teuthology.orchestra.run.smithi150.stdout:4232 read 0xa5a5008 thru 0xa5af1de (0xa1d7 bytes) 2024-09-21T23:11:25.005 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.005 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.005 INFO:teuthology.orchestra.run.smithi150.stdout:4236 write 0x8cfb707 thru 0x8cff9b2 (0x42ac bytes) 2024-09-21T23:11:25.010 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.010 INFO:teuthology.orchestra.run.smithi150.stdout:4239 read 0x1d957bb thru 0x1d9fe76 (0xa6bc bytes) 2024-09-21T23:11:25.011 INFO:teuthology.orchestra.run.smithi150.stdout:4241 punch from 0xcaa64c to 0xcac67d, (0x2031 bytes) 2024-09-21T23:11:25.012 INFO:teuthology.orchestra.run.smithi150.stdout:4242 write 0x1303901 thru 0x130fb06 (0xc206 bytes) 2024-09-21T23:11:25.018 INFO:teuthology.orchestra.run.smithi150.stdout:4243 write 0x7c1465 thru 0x7c2189 (0xd25 bytes) 2024-09-21T23:11:25.033 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.033 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.034 INFO:teuthology.orchestra.run.smithi150.stdout:4249 write 0xe52fae8 thru 0xe53f108 (0xf621 bytes) 2024-09-21T23:11:25.041 INFO:teuthology.orchestra.run.smithi150.stdout:4250 writesame 0xd7ea675 thru 0xd7f30ad data_size 0x1(0x8a39 bytes) 2024-09-21T23:11:25.062 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:25.062 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:25.062 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:25.059+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:25.076 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:25.094 INFO:teuthology.orchestra.run.smithi150.stdout:4251 punch from 0x9c030aa to 0x9c07171, (0x40c7 bytes) 2024-09-21T23:11:25.094 INFO:teuthology.orchestra.run.smithi150.stdout:4252 write 0x7e6f46f thru 0x7e7af81 (0xbb13 bytes) 2024-09-21T23:11:25.101 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.102 INFO:teuthology.orchestra.run.smithi150.stdout:4254 punch from 0xab8e875 to 0xab94ba7, (0x6332 bytes) 2024-09-21T23:11:25.102 INFO:teuthology.orchestra.run.smithi150.stdout:4255 punch from 0xc3bd847 to 0xc3c5fc4, (0x877d bytes) 2024-09-21T23:11:25.102 INFO:teuthology.orchestra.run.smithi150.stdout:4256 read 0xd2227c1 thru 0xd22738e (0x4bce bytes) 2024-09-21T23:11:25.102 INFO:teuthology.orchestra.run.smithi150.stdout:4257 write 0xdca8091 thru 0xdcb2e59 (0xadc9 bytes) 2024-09-21T23:11:25.108 INFO:teuthology.orchestra.run.smithi150.stdout:4258 punch from 0x991c8fb to 0x99298cb, (0xcfd0 bytes) 2024-09-21T23:11:25.127 INFO:teuthology.orchestra.run.smithi150.stdout:4259 trunc from 0xed84f1c to 0x1bf4601 2024-09-21T23:11:25.138 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:25.159 INFO:teuthology.orchestra.run.smithi150.stdout:4260 read 0x1b75fdb thru 0x1b7b526 (0x554c bytes) 2024-09-21T23:11:25.159 INFO:teuthology.orchestra.run.smithi150.stdout:4261 read 0x17e6886 thru 0x17ea9d0 (0x414b bytes) 2024-09-21T23:11:25.163 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:25.158+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:25.171 INFO:teuthology.orchestra.run.smithi150.stdout:4262 write 0x1ea576f thru 0x1ea9f0d (0x479f bytes) 2024-09-21T23:11:25.186 INFO:teuthology.orchestra.run.smithi150.stdout:4264 trunc from 0x1ea9f0e to 0xa5ccaec 2024-09-21T23:11:25.197 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.198 INFO:teuthology.orchestra.run.smithi150.stdout:4266 read 0x772a371 thru 0x7732e90 (0x8b20 bytes) 2024-09-21T23:11:25.198 INFO:teuthology.orchestra.run.smithi150.stdout:4267 punch from 0x36044 to 0x434f8, (0xd4b4 bytes) 2024-09-21T23:11:25.205 INFO:teuthology.orchestra.run.smithi150.stdout:4270 trunc from 0xa5ccaec to 0x54958cf 2024-09-21T23:11:25.216 INFO:teuthology.orchestra.run.smithi150.stdout:4271 punch from 0x16b5637 to 0x16b7a50, (0x2419 bytes) 2024-09-21T23:11:25.217 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.230 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:25.230 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:25.232 INFO:teuthology.orchestra.run.smithi150.stdout:4273 writesame 0x66198f5 thru 0x661aa92 data_size 0xb(0x119e bytes) 2024-09-21T23:11:25.232 INFO:teuthology.orchestra.run.smithi150.stdout:4274 read 0x131ccbc thru 0x1327ece (0xb213 bytes) 2024-09-21T23:11:25.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:25.251 INFO:teuthology.orchestra.run.smithi150.stdout:4275 write 0x9f4c8cd thru 0x9f4cea1 (0x5d5 bytes) 2024-09-21T23:11:25.274 INFO:teuthology.orchestra.run.smithi150.stdout:4276 writesame 0xe6e1538 thru 0xe6eee59 data_size 0x1(0xd922 bytes) 2024-09-21T23:11:25.274 INFO:teuthology.orchestra.run.smithi150.stdout:4277 writesame 0xb3044f9 thru 0xb310ccd data_size 0x1(0xc7d5 bytes) 2024-09-21T23:11:25.274 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:25.271+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:25.296 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:25.296 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:25.305 INFO:teuthology.orchestra.run.smithi150.stdout:4278 write 0x8ac852c thru 0x8aca5e5 (0x20ba bytes) 2024-09-21T23:11:25.310 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.310 INFO:teuthology.orchestra.run.smithi150.stdout:4280 read 0x53679de thru 0x5376907 (0xef2a bytes) 2024-09-21T23:11:25.310 INFO:teuthology.orchestra.run.smithi150.stdout:4281 read 0x65e9ee4 thru 0x65ec8c1 (0x29de bytes) 2024-09-21T23:11:25.316 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:25.316 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:25.319 INFO:teuthology.orchestra.run.smithi150.stdout:4283 trunc from 0xe6eee5a to 0x84da0ac 2024-09-21T23:11:25.335 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:25.336 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:25.336 INFO:teuthology.orchestra.run.smithi150.stdout:4284 write 0x274932c thru 0x2755763 (0xc438 bytes) 2024-09-21T23:11:25.343 INFO:teuthology.orchestra.run.smithi150.stdout:4285 trunc from 0x84da0ac to 0x523b02d 2024-09-21T23:11:25.353 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:25.353 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:25.354 INFO:teuthology.orchestra.run.smithi150.stdout:4287 write 0xb5e33f thru 0xb5fd6f (0x1a31 bytes) 2024-09-21T23:11:25.359 INFO:teuthology.orchestra.run.smithi150.stdout:4291 write 0x7caf8 thru 0x7fddc (0x32e5 bytes) 2024-09-21T23:11:25.363 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:25.359+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:25.372 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:25.376 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:25.377 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:25.380 INFO:teuthology.orchestra.run.smithi150.stdout:4295 trunc from 0x523b02d to 0xaacc6c3 2024-09-21T23:11:25.383 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:25.383 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:25.383 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:25.383 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:25.383 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:25.391 INFO:teuthology.orchestra.run.smithi150.stdout:4296 write 0x979c7a1 thru 0x97a4217 (0x7a77 bytes) 2024-09-21T23:11:25.396 INFO:teuthology.orchestra.run.smithi150.stdout:4297 write 0x80436f8 thru 0x804f445 (0xbd4e bytes) 2024-09-21T23:11:25.398 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:25.399 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:25.399 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:25.416 INFO:teuthology.orchestra.run.smithi150.stdout:4299 writesame 0xe3e375a thru 0xe3f0bfd data_size 0x3e(0xd4a4 bytes) 2024-09-21T23:11:25.420 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:25.420 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:25.427 INFO:teuthology.orchestra.run.smithi150.stdout:4300 write 0xed9b655 thru 0xeda1773 (0x611f bytes) 2024-09-21T23:11:25.443 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:25.444 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:25.451 INFO:teuthology.orchestra.run.smithi150.stdout:4301 trunc from 0xeda1774 to 0x165e8b2 2024-09-21T23:11:25.465 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:25.462+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:25.474 INFO:teuthology.orchestra.run.smithi150.stdout:4302 punch from 0x5535c9 to 0x559030, (0x5a67 bytes) 2024-09-21T23:11:25.504 INFO:teuthology.orchestra.run.smithi150.stdout:4304 write 0xd87259d thru 0xd875d42 (0x37a6 bytes) 2024-09-21T23:11:25.515 INFO:teuthology.orchestra.run.smithi150.stdout:4308 trunc from 0xd875d43 to 0x91b4832 2024-09-21T23:11:25.534 INFO:teuthology.orchestra.run.smithi150.stdout:4309 writesame 0x1912be0 thru 0x191fadb data_size 0x1a(0xcefc bytes) 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:25.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.057698031999998, 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.057644594000003, 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:11:25.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.057593619000002, 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:11:25.559 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:11:25.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.563 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.057542687999998, 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:11:25.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:11:25.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:11:25.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.057487377000001, 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:11:25.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:11:25.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.057418462000001, 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:11:25.572 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:11:25.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.576 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39835 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8314880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.052781109000001, 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01950789, 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39835 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496463+0000", 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496465+0000", 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496473+0000", 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496494+0000", 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1552e-05 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496510+0000", 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5537e-05 2024-09-21T23:11:25.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515291+0000", 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018780866 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515554+0000", 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000263634 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.515971+0000", 2024-09-21T23:11:25.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041722800000000001 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.052712968000002, 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:11:25.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:11:25.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.566559456999997, 2024-09-21T23:11:25.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41333 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9322496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.565506081999999, 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019340303, 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41333 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983739+0000", 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:11:25.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983746+0000", 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983762+0000", 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5892999999999998e-05 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983770+0000", 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:11:25.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002868+0000", 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019097592 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.002936+0000", 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8398999999999993e-05 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003079+0000", 2024-09-21T23:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142371 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41334 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9379840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.565459715000003, 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019648709, 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41334 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983785+0000", 2024-09-21T23:11:25.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983786+0000", 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983790+0000", 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983803+0000", 2024-09-21T23:11:25.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2658e-05 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983812+0000", 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4259999999999992e-06 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003118+0000", 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019305866000000001 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003228+0000", 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010946900000000001 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003434+0000", 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000205987 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.598 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.565331536000002, 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:25.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:11:25.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.565245740999998, 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.603 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41360 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9617408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.564701925999998, 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019718395999999999, 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.608 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41360 2024-09-21T23:11:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984543+0000", 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984545+0000", 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.103e-06 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984556+0000", 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1585e-05 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984586+0000", 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9454e-05 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984605+0000", 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9062000000000001e-05 2024-09-21T23:11:25.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004079+0000", 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019474114000000001 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004151+0000", 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2291000000000003e-05 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004261+0000", 2024-09-21T23:11:25.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109787 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.564604146000001, 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:11:25.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:11:25.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.564529272000001, 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.617 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.621 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.564468494000003, 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.625 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.564432175999997, 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.626 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.759970381999999, 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:11:25.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:11:25.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:11:25.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:25.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 26.243432215999999, 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.635 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:11:25.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:11:25.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:25.640 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:25.641 INFO:teuthology.orchestra.run.smithi150.stdout:4312 write 0xb19f715 thru 0xb1aa710 (0xaffc bytes) 2024-09-21T23:11:25.641 INFO:teuthology.orchestra.run.smithi150.stdout:4313 punch from 0x8f85e7f to 0x8f85f7d, (0xfe bytes) 2024-09-21T23:11:25.641 INFO:teuthology.orchestra.run.smithi150.stdout:4316 read 0x2cb9abb thru 0x2cc3640 (0x9b86 bytes) 2024-09-21T23:11:25.641 INFO:teuthology.orchestra.run.smithi150.stdout:4317 writesame 0x37db83 thru 0x38a813 data_size 0x1(0xcc91 bytes) 2024-09-21T23:11:25.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:4318 write 0xab73e9d thru 0xab7c5b5 (0x8719 bytes) 2024-09-21T23:11:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:4319 read 0xaefb193 thru 0xaf08731 (0xd59f bytes) 2024-09-21T23:11:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:4320 write 0x77e3b87 thru 0x77f11cb (0xd645 bytes) 2024-09-21T23:11:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:4323 write 0xd1f31f3 thru 0xd200e6c (0xdc7a bytes) 2024-09-21T23:11:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:4324 read 0x40bc82c thru 0x40caddc (0xe5b1 bytes) 2024-09-21T23:11:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:4326 clone 40 order 23 su 2097152 sc 5 2024-09-21T23:11:25.643 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:25.639+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:25.742 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:25.738+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:25.810 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:25.810 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:25.810 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:25.810 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:25.824 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:25.868 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:25.867+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:25.944 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:25.942+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:25.948 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:25.948 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:25.948 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:25.962 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:26.056 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:26.055+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.057482230000005, 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:11:26.135 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:11:26.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:11:26.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.801930735, 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.141 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:11:26.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.145 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.8019065149999998, 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.801860129, 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.8018449059999999, 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.158 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77015 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.79072611, 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059342726999999998, 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77015 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341443+0000", 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341445+0000", 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341453+0000", 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4869999999999996e-06 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341458+0000", 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400574+0000", 2024-09-21T23:11:26.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059115684000000002 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400680+0000", 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010637399999999999 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400785+0000", 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001047 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.162 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7906781519999999, 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.163 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7906554249999997, 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.167 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:26.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:26.171 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7906268279999997, 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:26.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:11:26.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:11:26.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7906146239999998, 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.176 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7905956330000001, 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.180 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:11:26.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7902613170000001, 2024-09-21T23:11:26.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:11:26.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.790217191, 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.189 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:11:26.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:11:26.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:11:26.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7901932340000002, 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:26.197 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.790153031, 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7790138410000003, 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:11:26.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7789918870000001, 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.206 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7789654690000001, 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:26.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:11:26.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.214 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7789391729999999, 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:11:26.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:11:26.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.778916561, 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:26.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:26.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:26.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:26.223 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:26.225 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:26.226 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:26.223+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:26.324 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:26.323+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:26.359 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:26.359 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:26.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:26.425 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:26.423+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:26.495 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:26.495 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:26.495 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:26.496 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:26.496 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:26.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:26.526 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:26.523+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:26.526 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:11:26.627 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:26.622+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:26.639 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:26.639 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:26.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:26.639 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:26.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.639 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.639 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.81369611, 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813625102, 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.647 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813560811, 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:11:26.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:26.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:11:26.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:11:26.651 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813530311999999, 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.652 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:11:26.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:11:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:11:26.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813382325999999, 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:11:26.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:11:26.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:11:26.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:11:26.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813325820999999, 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.660 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.663 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813271245999999, 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.664 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:11:26.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.667 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813216027999999, 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.668 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:11:26.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:11:26.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813161399, 2024-09-21T23:11:26.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:11:26.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.676 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.813042249, 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:11:26.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812964815000001, 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812899193, 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.685 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812834864999999, 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:26.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:26.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:11:26.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:11:26.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:11:26.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:11:26.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:11:26.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:11:26.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:11:26.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:11:26.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:11:26.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:11:26.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:11:26.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:11:26.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:11:26.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:11:26.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.698 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.699 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.699 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:26.699 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812773777, 2024-09-21T23:11:26.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:11:26.699 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.699 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.700 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.700 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:11:26.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.700 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:26.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:26.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:11:26.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:26.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:11:26.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:11:26.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:11:26.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:11:26.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812713312, 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.707 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:26.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.711 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812649113999999, 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812575106000001, 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.716 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:11:26.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:11:26.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:11:26.720 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.81253768, 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.721 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:11:26.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:11:26.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812491158, 2024-09-21T23:11:26.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:11:26.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:11:26.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:26.729 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.812431663, 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:26.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:26.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.733 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.734 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.734 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:26.734 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:26.734 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:26.735 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone39 from 0xd200e6d (overlap 0x165e8b2) to 0x6b5f68 2024-09-21T23:11:26.735 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:26.731+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:26.736 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:26.834 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-21T23:11:26.835 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:26.831+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:26.837 INFO:teuthology.orchestra.run.smithi150.stdout:4327 read 0x158416b thru 0x1585f0d (0x1da3 bytes) 2024-09-21T23:11:26.838 INFO:teuthology.orchestra.run.smithi150.stdout:4328 writesame 0x38a0867 thru 0x38a75de data_size 0x71(0x6d78 bytes) 2024-09-21T23:11:26.852 INFO:teuthology.orchestra.run.smithi150.stdout:4329 writesame 0x3c0ad21 thru 0x3c1994c data_size 0xa(0xec2c bytes) 2024-09-21T23:11:26.881 INFO:teuthology.orchestra.run.smithi150.stdout:4330 trunc from 0xd200e6d to 0x3a3503 2024-09-21T23:11:26.911 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:26.911 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:26.911 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:26.911 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:26.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:26.933 INFO:teuthology.orchestra.run.smithi150.stdout:4331 write 0x5f33c0 thru 0x5f6052 (0x2c93 bytes) 2024-09-21T23:11:26.936 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:26.931+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:26.938 INFO:teuthology.orchestra.run.smithi150.stdout:4332 read 0x4a2493 thru 0x4aff28 (0xda96 bytes) 2024-09-21T23:11:26.949 INFO:teuthology.orchestra.run.smithi150.stdout:4333 trunc from 0x5f6053 to 0x4bdbc26 2024-09-21T23:11:26.959 INFO:teuthology.orchestra.run.smithi150.stdout:4335 read 0x2b4a643 thru 0x2b581d4 (0xdb92 bytes) 2024-09-21T23:11:26.960 INFO:teuthology.orchestra.run.smithi150.stdout:4337 read 0x2b183c thru 0x2bae4c (0x9611 bytes) 2024-09-21T23:11:26.961 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:26.961 INFO:teuthology.orchestra.run.smithi150.stdout:4340 write 0x2339af5 thru 0x233bd12 (0x221e bytes) 2024-09-21T23:11:26.964 INFO:teuthology.orchestra.run.smithi150.stdout:4341 punch from 0x22c925b to 0x22d7bf7, (0xe99c bytes) 2024-09-21T23:11:26.983 INFO:teuthology.orchestra.run.smithi150.stdout:4343 writesame 0x7c5f886 thru 0x7c61903 data_size 0x1(0x207e bytes) 2024-09-21T23:11:26.983 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:26.983 INFO:teuthology.orchestra.run.smithi150.stdout:4346 read 0x2932812 thru 0x2932a5f (0x24e bytes) 2024-09-21T23:11:26.983 INFO:teuthology.orchestra.run.smithi150.stdout:4347 write 0x5f36656 thru 0x5f3f554 (0x8eff bytes) 2024-09-21T23:11:26.989 INFO:teuthology.orchestra.run.smithi150.stdout:4349 punch from 0x740d630 to 0x7411529, (0x3ef9 bytes) 2024-09-21T23:11:26.994 INFO:teuthology.orchestra.run.smithi150.stdout:4350 trunc from 0x7c61904 to 0x96644ab 2024-09-21T23:11:27.024 INFO:teuthology.orchestra.run.smithi150.stdout:4351 write 0xc66e577 thru 0xc6787ec (0xa276 bytes) 2024-09-21T23:11:27.031 INFO:teuthology.orchestra.run.smithi150.stdout:4352 write 0x45ec66a thru 0x45ef5ae (0x2f45 bytes) 2024-09-21T23:11:27.036 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:27.034+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:27.038 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:27.038 INFO:teuthology.orchestra.run.smithi150.stdout:4355 read 0x43cabb7 thru 0x43cd99e (0x2de8 bytes) 2024-09-21T23:11:27.038 INFO:teuthology.orchestra.run.smithi150.stdout:4357 read 0x5077a29 thru 0x5079a7b (0x2053 bytes) 2024-09-21T23:11:27.053 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:27.053 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:27.053 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:27.053 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:27.054 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:27.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:27.069 INFO:teuthology.orchestra.run.smithi150.stdout:4358 trunc from 0xc6787ed to 0x653134 2024-09-21T23:11:27.093 INFO:teuthology.orchestra.run.smithi150.stdout:4359 read 0x3c6e79 thru 0x3cdae8 (0x6c70 bytes) 2024-09-21T23:11:27.094 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:27.125 INFO:teuthology.orchestra.run.smithi150.stdout:4362 trunc from 0x653134 to 0xd533aa9 2024-09-21T23:11:27.137 INFO:teuthology.orchestra.run.smithi150.stdout:4363 read 0x4b9cbee thru 0x4ba4c9f (0x80b2 bytes) 2024-09-21T23:11:27.137 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:27.137 INFO:teuthology.orchestra.run.smithi150.stdout:4365 read 0xb5bf3dc thru 0xb5cc0fc (0xcd21 bytes) 2024-09-21T23:11:27.137 INFO:teuthology.orchestra.run.smithi150.stdout:4368 read 0x48c201d thru 0x48cd497 (0xb47b bytes) 2024-09-21T23:11:27.137 INFO:teuthology.orchestra.run.smithi150.stdout:4369 read 0xd29a3a8 thru 0xd2a8720 (0xe379 bytes) 2024-09-21T23:11:27.137 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:27.135+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:27.137 INFO:teuthology.orchestra.run.smithi150.stdout:4373 write 0x5e3ddab thru 0x5e4c728 (0xe97e bytes) 2024-09-21T23:11:27.157 INFO:teuthology.orchestra.run.smithi150.stdout:4374 trunc from 0xd533aa9 to 0x7a68386 2024-09-21T23:11:27.169 INFO:teuthology.orchestra.run.smithi150.stdout:4375 writesame 0x543f04c thru 0x54491dd data_size 0x1(0xa192 bytes) 2024-09-21T23:11:27.212 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:27.212 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:27.212 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:11:27.213 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:11:27.213 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:11:27.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:11:27.231 INFO:teuthology.orchestra.run.smithi150.stdout:4376 writesame 0xd3885d3 thru 0xd39695b data_size 0x1(0xe389 bytes) 2024-09-21T23:11:27.232 INFO:teuthology.orchestra.run.smithi150.stdout:4377 write 0x3338224 thru 0x33461be (0xdf9b bytes) 2024-09-21T23:11:27.237 INFO:teuthology.orchestra.run.smithi150.stdout:4378 write 0x4838274 thru 0x4841265 (0x8ff2 bytes) 2024-09-21T23:11:27.237 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:27.235+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:27.242 INFO:teuthology.orchestra.run.smithi150.stdout:4379 read 0xf54633 thru 0xf62ca9 (0xe677 bytes) 2024-09-21T23:11:27.243 INFO:teuthology.orchestra.run.smithi150.stdout:4380 punch from 0x271d8a6 to 0x2720e26, (0x3580 bytes) 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:27.244 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.514901590000001, 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:27.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:27.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:11:27.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:11:27.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8675880969999996, 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:11:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8675554219999997, 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:11:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8675320869999998, 2024-09-21T23:11:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:11:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8674995220000001, 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:11:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8674119000000005, 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:11:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8672931530000003, 2024-09-21T23:11:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:11:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:11:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8672683880000003, 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:11:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:11:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8672472410000003, 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:11:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8671895440000004, 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.867154663, 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8671291380000001, 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:11:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:11:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8670579519999997, 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:11:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:11:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:11:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8670221060000003, 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:11:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:11:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.867005013, 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:11:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:11:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8669890020000004, 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:11:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:11:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8669317540000003, 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:11:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:11:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8668503889999997, 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:11:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:11:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.8668188390000005, 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.866671942, 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:11:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:27.349 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:27.351 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4381 write 0xdd6d1e7 thru 0xdd77daa (0xabc4 bytes) 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4383 read 0xb50c630 thru 0xb518d57 (0xc728 bytes) 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4384 punch from 0x84667b3 to 0x8472a48, (0xc295 bytes) 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4385 read 0x7bf5169 thru 0x7bf6e51 (0x1ce9 bytes) 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4388 write 0xedc855b thru 0xedd52c4 (0xcd6a bytes) 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4389 trunc from 0xedd52c5 to 0x56ec7e7 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4391 read 0x38c6cb7 thru 0x38d22cb (0xb615 bytes) 2024-09-21T23:11:27.352 INFO:teuthology.orchestra.run.smithi150.stdout:4392 clone 41 order 22 su 32768 sc 10 2024-09-21T23:11:27.353 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:27.347+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:27.354 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:27.354 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "" 2024-09-21T23:11:27.354 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:27.451 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:27.446+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:27.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:27.482 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:27.482 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:27.482 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:27.496 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:27.573 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:11:27.584 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:27.582+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:27.624 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:27.624 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:27.642 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:27.642 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:27.653 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:27.650+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:27.659 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:27.659 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:27.668 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:27.668 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:27.668 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:27.668 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:27.668 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:27.677 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:27.677 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:27.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:27.698 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:27.698 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:27.719 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone40 from 0x56ec7e7 (overlap 0x3a3503) to 0x0 2024-09-21T23:11:27.721 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:27.745 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:27.745 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:27.767 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:27.767 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:27.776 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:27.774+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:27.854 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:27.851+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:27.857 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:27.857 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:27.871 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:27.948 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-21T23:11:27.956 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:27.951+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:27.978 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:27.978 INFO:teuthology.orchestra.run.smithi150.stdout:4396 read 0x83121e thru 0x83a9aa (0x978d bytes) 2024-09-21T23:11:27.980 INFO:teuthology.orchestra.run.smithi150.stdout:4397 write 0x12d7944 thru 0x12e22af (0xa96c bytes) 2024-09-21T23:11:28.044 INFO:teuthology.orchestra.run.smithi150.stdout:4398 punch from 0x109b7b to 0x109f11, (0x396 bytes) 2024-09-21T23:11:28.045 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:28.045 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:28.057 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:28.055+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:28.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:28.073 INFO:teuthology.orchestra.run.smithi150.stdout:4399 writesame 0xc7100ae thru 0xc7168e2 data_size 0x7(0x6835 bytes) 2024-09-21T23:11:28.073 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.073 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.073 INFO:teuthology.orchestra.run.smithi150.stdout:4402 write 0xc6b98b9 thru 0xc6c2684 (0x8dcc bytes) 2024-09-21T23:11:28.082 INFO:teuthology.orchestra.run.smithi150.stdout:4403 read 0x7aea77c thru 0x7af64e2 (0xbd67 bytes) 2024-09-21T23:11:28.082 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.082 INFO:teuthology.orchestra.run.smithi150.stdout:4406 read 0xae1da53 thru 0xae29533 (0xbae1 bytes) 2024-09-21T23:11:28.082 INFO:teuthology.orchestra.run.smithi150.stdout:4407 read 0xc55bff4 thru 0xc56159d (0x55aa bytes) 2024-09-21T23:11:28.082 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.082 INFO:teuthology.orchestra.run.smithi150.stdout:4409 writesame 0x94524ae thru 0x945f2f4 data_size 0x1(0xce47 bytes) 2024-09-21T23:11:28.131 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.131 INFO:teuthology.orchestra.run.smithi150.stdout:4413 read 0x336544f thru 0x336c253 (0x6e05 bytes) 2024-09-21T23:11:28.132 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.132 INFO:teuthology.orchestra.run.smithi150.stdout:4415 read 0x9a2e217 thru 0x9a39dc5 (0xbbaf bytes) 2024-09-21T23:11:28.156 INFO:teuthology.orchestra.run.smithi150.stdout:4416 trunc from 0xc7168e3 to 0x3335e86 2024-09-21T23:11:28.157 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:28.155+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:28.184 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:28.185 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:28.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:28.258 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:28.254+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:28.271 INFO:teuthology.orchestra.run.smithi150.stdout:4417 read 0x28cfe0 thru 0x29ba5f (0xea80 bytes) 2024-09-21T23:11:28.273 INFO:teuthology.orchestra.run.smithi150.stdout:4418 read 0x7a95ff thru 0x7afa5f (0x6461 bytes) 2024-09-21T23:11:28.283 INFO:teuthology.orchestra.run.smithi150.stdout:4422 writesame 0x4d8d493 thru 0x4d993fe data_size 0x2(0xbf6c bytes) 2024-09-21T23:11:28.285 INFO:teuthology.orchestra.run.smithi150.stdout:4423 writesame 0x402160e thru 0x402ece7 data_size 0x2(0xd6da bytes) 2024-09-21T23:11:28.298 INFO:teuthology.orchestra.run.smithi150.stdout:4424 punch from 0x4680bb8 to 0x4682e2a, (0x2272 bytes) 2024-09-21T23:11:28.306 INFO:teuthology.orchestra.run.smithi150.stdout:4425 trunc from 0x4d993ff to 0x88ae66e 2024-09-21T23:11:28.315 INFO:teuthology.orchestra.run.smithi150.stdout:4427 read 0x2dec91c thru 0x2df9979 (0xd05e bytes) 2024-09-21T23:11:28.316 INFO:teuthology.orchestra.run.smithi150.stdout:4428 write 0x4021767 thru 0x402216f (0xa09 bytes) 2024-09-21T23:11:28.326 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:28.326 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:28.334 INFO:teuthology.orchestra.run.smithi150.stdout:4429 write 0x932b0ff thru 0x933adad (0xfcaf bytes) 2024-09-21T23:11:28.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:28.344 INFO:teuthology.orchestra.run.smithi150.stdout:4431 write 0x6fafc73 thru 0x6fafd6a (0xf8 bytes) 2024-09-21T23:11:28.348 INFO:teuthology.orchestra.run.smithi150.stdout:4432 punch from 0x4da4cb6 to 0x4dace0d, (0x8157 bytes) 2024-09-21T23:11:28.361 INFO:teuthology.orchestra.run.smithi150.stdout:4433 writesame 0xaec14b1 thru 0xaec5af6 data_size 0x23(0x4646 bytes) 2024-09-21T23:11:28.362 INFO:teuthology.orchestra.run.smithi150.stdout:4434 read 0xab094f0 thru 0xab1045c (0x6f6d bytes) 2024-09-21T23:11:28.365 INFO:teuthology.orchestra.run.smithi150.stdout:4435 trunc from 0xaec5af7 to 0xc8981e1 2024-09-21T23:11:28.374 INFO:teuthology.orchestra.run.smithi150.stdout:4437 write 0x3e11760 thru 0x3e21222 (0xfac3 bytes) 2024-09-21T23:11:28.377 INFO:teuthology.orchestra.run.smithi150.stdout:4438 read 0x161b4fb thru 0x161bed8 (0x9de bytes) 2024-09-21T23:11:28.378 INFO:teuthology.orchestra.run.smithi150.stdout:4439 punch from 0xc035b9b to 0xc039494, (0x38f9 bytes) 2024-09-21T23:11:28.384 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:28.382+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:28.392 INFO:teuthology.orchestra.run.smithi150.stdout:4440 trunc from 0xc8981e1 to 0x553b638 2024-09-21T23:11:28.409 INFO:teuthology.orchestra.run.smithi150.stdout:4441 trunc from 0x553b638 to 0x4479e56 2024-09-21T23:11:28.454 INFO:teuthology.orchestra.run.smithi150.stdout:4442 write 0xb90a3f6 thru 0xb917910 (0xd51b bytes) 2024-09-21T23:11:28.460 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:28.458+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:28.463 INFO:teuthology.orchestra.run.smithi150.stdout:4443 read 0xacff1fb thru 0xad08d08 (0x9b0e bytes) 2024-09-21T23:11:28.479 INFO:teuthology.orchestra.run.smithi150.stdout:4444 write 0xee2bb57 thru 0xee30fae (0x5458 bytes) 2024-09-21T23:11:28.489 INFO:teuthology.orchestra.run.smithi150.stdout:4445 write 0x519256 thru 0x521eb5 (0x8c60 bytes) 2024-09-21T23:11:28.489 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:28.489 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:28.493 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.493 INFO:teuthology.orchestra.run.smithi150.stdout:4447 punch from 0x902ae8 to 0x908c5f, (0x6177 bytes) 2024-09-21T23:11:28.493 INFO:teuthology.orchestra.run.smithi150.stdout:4448 write 0xc8379eb thru 0xc841b3f (0xa155 bytes) 2024-09-21T23:11:28.502 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:28.523 INFO:teuthology.orchestra.run.smithi150.stdout:4450 trunc from 0xee30faf to 0x1640ef4 2024-09-21T23:11:28.561 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:28.558+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:28.639 INFO:teuthology.orchestra.run.smithi150.stdout:4451 write 0x7f4eb7 thru 0x7fac5c (0x5da6 bytes) 2024-09-21T23:11:28.641 INFO:teuthology.orchestra.run.smithi150.stdout:4452 read 0xe520f thru 0xf2957 (0xd749 bytes) 2024-09-21T23:11:28.642 INFO:teuthology.orchestra.run.smithi150.stdout:4453 punch from 0x478390 to 0x47fb31, (0x77a1 bytes) 2024-09-21T23:11:28.647 INFO:teuthology.orchestra.run.smithi150.stdout:4454 trunc from 0x1640ef4 to 0x4869724 2024-09-21T23:11:28.662 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:28.659+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:28.669 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:28.669 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:28.680 INFO:teuthology.orchestra.run.smithi150.stdout:4455 writesame 0xc50a7e9 thru 0xc50ff28 data_size 0x10(0x5740 bytes) 2024-09-21T23:11:28.680 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:28.680 INFO:teuthology.orchestra.run.smithi150.stdout:4457 punch from 0xbdc055a to 0xbdca07b, (0x9b21 bytes) 2024-09-21T23:11:28.680 INFO:teuthology.orchestra.run.smithi150.stdout:4458 write 0xa5dfe62 thru 0xa5efadf (0xfc7e bytes) 2024-09-21T23:11:28.686 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:28.691 INFO:teuthology.orchestra.run.smithi150.stdout:4459 trunc from 0xc50ff29 to 0xb2e81c0 2024-09-21T23:11:28.704 INFO:teuthology.orchestra.run.smithi150.stdout:4460 punch from 0x5163d54 to 0x516c0a7, (0x8353 bytes) 2024-09-21T23:11:28.704 INFO:teuthology.orchestra.run.smithi150.stdout:4461 punch from 0x3f6e61a to 0x3f774f9, (0x8edf bytes) 2024-09-21T23:11:28.713 INFO:teuthology.orchestra.run.smithi150.stdout:4462 write 0xbbb8f81 thru 0xbbc354f (0xa5cf bytes) 2024-09-21T23:11:28.731 INFO:teuthology.orchestra.run.smithi150.stdout:4463 write 0xecb1177 thru 0xecb9ad3 (0x895d bytes) 2024-09-21T23:11:28.756 INFO:teuthology.orchestra.run.smithi150.stdout:4464 trunc from 0xecb9ad4 to 0x423ef14 2024-09-21T23:11:28.763 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:28.759+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:28.777 INFO:teuthology.orchestra.run.smithi150.stdout:4466 read 0x1d228b3 thru 0x1d24469 (0x1bb7 bytes) 2024-09-21T23:11:28.797 INFO:teuthology.orchestra.run.smithi150.stdout:4467 write 0xbc4ee4c thru 0xbc50343 (0x14f8 bytes) 2024-09-21T23:11:28.805 INFO:teuthology.orchestra.run.smithi150.stdout:4468 clone 42 order 22 su 1048576 sc 5 2024-09-21T23:11:28.825 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:28.825 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:28.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:28.864 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:28.862+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:28.964 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:28.962+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:29.005 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:29.005 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:29.019 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:29.066 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:29.062+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:29.140 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:29.140 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:29.154 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:29.200 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:29.199+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:29.267 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:29.263+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:29.272 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:29.272 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:29.286 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:29.376 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:29.375+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:29.454 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:29.454 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:29.469 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:29.469 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:29.463+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:29.568 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:29.567+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:29.597 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:11:29.645 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:29.645 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:29.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:29.669 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:29.666+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:29.763 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone41 from 0xbc50344 (overlap 0x1640ef4) to 0x8e6c61 2024-09-21T23:11:29.770 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:29.766+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:29.785 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:29.810 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:29.810 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:29.828 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:29.871 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:29.866+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:29.944 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:29.944 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:29.951 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-21T23:11:29.953 INFO:teuthology.orchestra.run.smithi150.stdout:4469 writesame 0xae343c thru 0xaed8b6 data_size 0x29(0xa47b bytes) 2024-09-21T23:11:29.968 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:29.968 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:29.971 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:29.967+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:29.985 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:29.985 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:29.989 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:29.989 INFO:teuthology.orchestra.run.smithi150.stdout:4473 write 0x9ad967d thru 0x9ae355b (0x9edf bytes) 2024-09-21T23:11:29.996 INFO:teuthology.orchestra.run.smithi150.stdout:4474 punch from 0x244e51b to 0x244e686, (0x16b bytes) 2024-09-21T23:11:29.996 INFO:teuthology.orchestra.run.smithi150.stdout:4475 punch from 0xb7aa6a8 to 0xb7b26d5, (0x802d bytes) 2024-09-21T23:11:29.996 INFO:teuthology.orchestra.run.smithi150.stdout:4476 write 0x6d04344 thru 0x6d130d8 (0xed95 bytes) 2024-09-21T23:11:30.003 INFO:teuthology.orchestra.run.smithi150.stdout:4478 read 0x1615393 thru 0x161db9c (0x880a bytes) 2024-09-21T23:11:30.003 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:30.003 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:30.004 INFO:teuthology.orchestra.run.smithi150.stdout:4479 write 0x434ead9 thru 0x435407d (0x55a5 bytes) 2024-09-21T23:11:30.011 INFO:teuthology.orchestra.run.smithi150.stdout:4481 read 0x8c31bf1 thru 0x8c38fbf (0x73cf bytes) 2024-09-21T23:11:30.011 INFO:teuthology.orchestra.run.smithi150.stdout:4482 writesame 0x191ad31 thru 0x191e9e9 data_size 0x1(0x3cb9 bytes) 2024-09-21T23:11:30.013 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:30.013 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:30.013 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:30.013 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:30.027 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:30.027 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:30.028 INFO:teuthology.orchestra.run.smithi150.stdout:4483 write 0x58ba5f0 thru 0x58c7d78 (0xd789 bytes) 2024-09-21T23:11:30.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:30.034 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.038 INFO:teuthology.orchestra.run.smithi150.stdout:4487 trunc from 0xbc50344 to 0x962106c 2024-09-21T23:11:30.048 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:30.048 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:30.063 INFO:teuthology.orchestra.run.smithi150.stdout:4488 punch from 0x171e794 to 0x171ed75, (0x5e1 bytes) 2024-09-21T23:11:30.063 INFO:teuthology.orchestra.run.smithi150.stdout:4489 punch from 0x2f671ea to 0x2f756ea, (0xe500 bytes) 2024-09-21T23:11:30.063 INFO:teuthology.orchestra.run.smithi150.stdout:4490 punch from 0x2d3b18 to 0x2d62dc, (0x27c4 bytes) 2024-09-21T23:11:30.063 INFO:teuthology.orchestra.run.smithi150.stdout:4491 punch from 0x11e28ed to 0x11e4154, (0x1867 bytes) 2024-09-21T23:11:30.072 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:30.072 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:30.072 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:30.071+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:30.073 INFO:teuthology.orchestra.run.smithi150.stdout:4492 writesame 0xbd1a9bf thru 0xbd27a6f data_size 0x5(0xd0b1 bytes) 2024-09-21T23:11:30.073 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.084 INFO:teuthology.orchestra.run.smithi150.stdout:4494 trunc from 0xbd27a70 to 0x44da50c 2024-09-21T23:11:30.095 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:30.095 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:30.118 INFO:teuthology.orchestra.run.smithi150.stdout:4495 write 0x1c34f95 thru 0x1c36541 (0x15ad bytes) 2024-09-21T23:11:30.150 INFO:teuthology.orchestra.run.smithi150.stdout:4496 write 0xe4dba0a thru 0xe4e85f4 (0xcbeb bytes) 2024-09-21T23:11:30.157 INFO:teuthology.orchestra.run.smithi150.stdout:4498 read 0xb779e70 thru 0xb783bec (0x9d7d bytes) 2024-09-21T23:11:30.157 INFO:teuthology.orchestra.run.smithi150.stdout:4499 read 0x19b3448 thru 0x19b834d (0x4f06 bytes) 2024-09-21T23:11:30.157 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.157 INFO:teuthology.orchestra.run.smithi150.stdout:4502 write 0x9f4c4d3 thru 0x9f590b9 (0xcbe7 bytes) 2024-09-21T23:11:30.163 INFO:teuthology.orchestra.run.smithi150.stdout:4503 read 0x719e846 thru 0x71a0e16 (0x25d1 bytes) 2024-09-21T23:11:30.170 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:30.170 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:30.170 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:30.170 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:30.170 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:30.174 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:30.171+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:30.181 INFO:teuthology.orchestra.run.smithi150.stdout:4504 trunc from 0xe4e85f5 to 0x19b78dd 2024-09-21T23:11:30.193 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:30.251 INFO:teuthology.orchestra.run.smithi150.stdout:4505 write 0xe369d9b thru 0xe36a084 (0x2ea bytes) 2024-09-21T23:11:30.257 INFO:teuthology.orchestra.run.smithi150.stdout:4506 writesame 0xbcc8846 thru 0xbcd2f10 data_size 0x2b(0xa6cb bytes) 2024-09-21T23:11:30.265 INFO:teuthology.orchestra.run.smithi150.stdout:4507 writesame 0xaa23d2c thru 0xaa2a27b data_size 0x8(0x6550 bytes) 2024-09-21T23:11:30.273 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.273 INFO:teuthology.orchestra.run.smithi150.stdout:4511 read 0x1b82118 thru 0x1b83e24 (0x1d0d bytes) 2024-09-21T23:11:30.274 INFO:teuthology.orchestra.run.smithi150.stdout:4512 read 0x528c03c thru 0x528e97f (0x2944 bytes) 2024-09-21T23:11:30.274 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:30.271+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:30.274 INFO:teuthology.orchestra.run.smithi150.stdout:4513 write 0x6b0dfc3 thru 0x6b119fc (0x3a3a bytes) 2024-09-21T23:11:30.281 INFO:teuthology.orchestra.run.smithi150.stdout:4514 punch from 0xf4c07a to 0xf4fd68, (0x3cee bytes) 2024-09-21T23:11:30.281 INFO:teuthology.orchestra.run.smithi150.stdout:4517 punch from 0xdc1b255 to 0xdc210de, (0x5e89 bytes) 2024-09-21T23:11:30.281 INFO:teuthology.orchestra.run.smithi150.stdout:4518 write 0x9e36493 thru 0x9e412c8 (0xae36 bytes) 2024-09-21T23:11:30.287 INFO:teuthology.orchestra.run.smithi150.stdout:4520 writesame 0x37adbb2 thru 0x37b0fe2 data_size 0x1(0x3431 bytes) 2024-09-21T23:11:30.309 INFO:teuthology.orchestra.run.smithi150.stdout:4521 read 0x8189a34 thru 0x818c8b4 (0x2e81 bytes) 2024-09-21T23:11:30.309 INFO:teuthology.orchestra.run.smithi150.stdout:4522 write 0xabcb525 thru 0xabd32ea (0x7dc6 bytes) 2024-09-21T23:11:30.316 INFO:teuthology.orchestra.run.smithi150.stdout:4523 punch from 0x3ff765f to 0x3ff8c76, (0x1617 bytes) 2024-09-21T23:11:30.316 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.316 INFO:teuthology.orchestra.run.smithi150.stdout:4532 write 0x8a2852e thru 0x8a2afc5 (0x2a98 bytes) 2024-09-21T23:11:30.323 INFO:teuthology.orchestra.run.smithi150.stdout:4534 read 0xa8cbb7e thru 0xa8dabcc (0xf04f bytes) 2024-09-21T23:11:30.335 INFO:teuthology.orchestra.run.smithi150.stdout:4536 write 0xe93bc15 thru 0xe93fb0e (0x3efa bytes) 2024-09-21T23:11:30.341 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.341 INFO:teuthology.orchestra.run.smithi150.stdout:4540 write 0x62ef17 thru 0x636173 (0x725d bytes) 2024-09-21T23:11:30.362 INFO:teuthology.orchestra.run.smithi150.stdout:4542 write 0xc010fd6 thru 0xc01ba8b (0xaab6 bytes) 2024-09-21T23:11:30.369 INFO:teuthology.orchestra.run.smithi150.stdout:4543 read 0x681535 thru 0x689e95 (0x8961 bytes) 2024-09-21T23:11:30.370 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:30.370 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:30.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:30.370 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:30.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 79.294965898000001, 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:11:30.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:30.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:30.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0394144030000003, 2024-09-21T23:11:30.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.380 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0393901830000001, 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:30.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:11:30.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:11:30.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0393437970000008, 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.388 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0393285740000007, 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:11:30.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:11:30.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:11:30.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:11:30.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77015 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0282097780000008, 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059342726999999998, 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77015 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341443+0000", 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341445+0000", 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341453+0000", 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4869999999999996e-06 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341458+0000", 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400574+0000", 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059115684000000002 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400680+0000", 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010637399999999999 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400785+0000", 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001047 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0281618199999993, 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:11:30.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0281390930000001, 2024-09-21T23:11:30.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:11:30.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:11:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.405 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.028110496, 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:11:30.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:11:30.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0280982919999992, 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:11:30.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:11:30.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.028079301, 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.027744985, 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.423 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0277008589999994, 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.427 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0276769019999996, 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:11:30.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:30.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0276366990000003, 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:11:30.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0164975090000006, 2024-09-21T23:11:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:11:30.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:11:30.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0164755549999995, 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:11:30.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:11:30.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:11:30.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0164491370000004, 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:11:30.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:11:30.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:30.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0164228410000007, 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:11:30.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.0164002290000003, 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:11:30.454 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:11:30.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:11:30.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:30.458 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:30.459 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:30.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:30.460 INFO:teuthology.orchestra.run.smithi150.stdout:4544 trunc from 0xe93fb0f to 0x1053d39 2024-09-21T23:11:30.461 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:30.455+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:30.502 INFO:teuthology.orchestra.run.smithi150.stdout:4545 write 0xeb516c3 thru 0xeb60c6e (0xf5ac bytes) 2024-09-21T23:11:30.507 INFO:teuthology.orchestra.run.smithi150.stdout:4547 write 0x39c4b29 thru 0x39cede6 (0xa2be bytes) 2024-09-21T23:11:30.511 INFO:teuthology.orchestra.run.smithi150.stdout:4548 write 0x63f58ca thru 0x63f71b1 (0x18e8 bytes) 2024-09-21T23:11:30.517 INFO:teuthology.orchestra.run.smithi150.stdout:4550 write 0xa60bed5 thru 0xa610bd0 (0x4cfc bytes) 2024-09-21T23:11:30.522 INFO:teuthology.orchestra.run.smithi150.stdout:4551 write 0x570e6d5 thru 0x571beab (0xd7d7 bytes) 2024-09-21T23:11:30.528 INFO:teuthology.orchestra.run.smithi150.stdout:4552 read 0xbebfb1a thru 0xbec496c (0x4e53 bytes) 2024-09-21T23:11:30.528 INFO:teuthology.orchestra.run.smithi150.stdout:4553 write 0xd1ec80 thru 0xd2ae3a (0xc1bb bytes) 2024-09-21T23:11:30.531 INFO:teuthology.orchestra.run.smithi150.stdout:4555 read 0x60e066a thru 0x60eec83 (0xe61a bytes) 2024-09-21T23:11:30.531 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.531 INFO:teuthology.orchestra.run.smithi150.stdout:4557 punch from 0x2bd3ad to 0x2be949, (0x159c bytes) 2024-09-21T23:11:30.531 INFO:teuthology.orchestra.run.smithi150.stdout:4558 read 0x1a9282d thru 0x1a93bd8 (0x13ac bytes) 2024-09-21T23:11:30.531 INFO:teuthology.orchestra.run.smithi150.stdout:4559 read 0xb2b3bc9 thru 0xb2bbbb6 (0x7fee bytes) 2024-09-21T23:11:30.531 INFO:teuthology.orchestra.run.smithi150.stdout:4561 write 0x6cd5640 thru 0x6cdc6f8 (0x70b9 bytes) 2024-09-21T23:11:30.536 INFO:teuthology.orchestra.run.smithi150.stdout:4562 write 0x53f579d thru 0x5404e1a (0xf67e bytes) 2024-09-21T23:11:30.541 INFO:teuthology.orchestra.run.smithi150.stdout:4563 punch from 0x423f422 to 0x4244006, (0x4be4 bytes) 2024-09-21T23:11:30.541 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.541 INFO:teuthology.orchestra.run.smithi150.stdout:4570 punch from 0xbef6287 to 0xbefe9d3, (0x874c bytes) 2024-09-21T23:11:30.542 INFO:teuthology.orchestra.run.smithi150.stdout:4571 punch from 0x2cab6fb to 0x2cb8351, (0xcc56 bytes) 2024-09-21T23:11:30.542 INFO:teuthology.orchestra.run.smithi150.stdout:4573 read 0x98ee670 thru 0x98eef61 (0x8f2 bytes) 2024-09-21T23:11:30.542 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:30.542 INFO:teuthology.orchestra.run.smithi150.stdout:4575 write 0x8348e9 thru 0x834bd6 (0x2ee bytes) 2024-09-21T23:11:30.543 INFO:teuthology.orchestra.run.smithi150.stdout:4578 writesame 0x71c15c2 thru 0x71c8cca data_size 0x1(0x7709 bytes) 2024-09-21T23:11:30.561 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:30.558+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:30.580 INFO:teuthology.orchestra.run.smithi150.stdout:4579 write 0x3e43523 thru 0x3e50f4d (0xda2b bytes) 2024-09-21T23:11:30.598 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:30.599 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:30.599 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:30.599 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:30.607 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:11:30.612 INFO:teuthology.orchestra.run.smithi150.stdout:4580 write 0xed0499a thru 0xed0bfb2 (0x7619 bytes) 2024-09-21T23:11:30.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:30.621 INFO:teuthology.orchestra.run.smithi150.stdout:4581 write 0x7cf6c79 thru 0x7cfd263 (0x65eb bytes) 2024-09-21T23:11:30.626 INFO:teuthology.orchestra.run.smithi150.stdout:4582 writesame 0x6d9fa26 thru 0x6dab4c2 data_size 0x2b(0xba9d bytes) 2024-09-21T23:11:30.631 INFO:teuthology.orchestra.run.smithi150.stdout:4584 write 0x49d7773 thru 0x49e38a2 (0xc130 bytes) 2024-09-21T23:11:30.635 INFO:teuthology.orchestra.run.smithi150.stdout:4585 punch from 0xc0ed75d to 0xc0f9b17, (0xc3ba bytes) 2024-09-21T23:11:30.658 INFO:teuthology.orchestra.run.smithi150.stdout:4586 trunc from 0xed0bfb3 to 0x3018564 2024-09-21T23:11:30.662 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:30.658+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:30.680 INFO:teuthology.orchestra.run.smithi150.stdout:4588 read 0x2066f1a thru 0x2067d8c (0xe73 bytes) 2024-09-21T23:11:30.695 INFO:teuthology.orchestra.run.smithi150.stdout:4589 trunc from 0x3018564 to 0x9905580 2024-09-21T23:11:30.716 INFO:teuthology.orchestra.run.smithi150.stdout:4591 writesame 0x9adb4a5 thru 0x9ae49fe data_size 0x7(0x955a bytes) 2024-09-21T23:11:30.716 INFO:teuthology.orchestra.run.smithi150.stdout:4592 write 0x8831635 thru 0x8839e8d (0x8859 bytes) 2024-09-21T23:11:30.722 INFO:teuthology.orchestra.run.smithi150.stdout:4595 write 0x600e6b8 thru 0x6012fde (0x4927 bytes) 2024-09-21T23:11:30.740 INFO:teuthology.orchestra.run.smithi150.stdout:4596 writesame 0xafac1c2 thru 0xafbb845 data_size 0x12(0xf684 bytes) 2024-09-21T23:11:30.740 INFO:teuthology.orchestra.run.smithi150.stdout:4598 writesame 0xa380c85 thru 0xa386263 data_size 0x13(0x55df bytes) 2024-09-21T23:11:30.746 INFO:teuthology.orchestra.run.smithi150.stdout:4599 writesame 0x69177d8 thru 0x6921bef data_size 0x59(0xa418 bytes) 2024-09-21T23:11:30.752 INFO:teuthology.orchestra.run.smithi150.stdout:4600 read 0x9fee215 thru 0x9ff8c4d (0xaa39 bytes) 2024-09-21T23:11:30.752 INFO:teuthology.orchestra.run.smithi150.stdout:4601 write 0x1b2db7f thru 0x1b31cfc (0x417e bytes) 2024-09-21T23:11:30.758 INFO:teuthology.orchestra.run.smithi150.stdout:4605 punch from 0x54ce99a to 0x54d978e, (0xadf4 bytes) 2024-09-21T23:11:30.763 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:30.759+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:30.771 INFO:teuthology.orchestra.run.smithi150.stdout:4606 write 0xbcc0de0 thru 0xbcc1bbe (0xddf bytes) 2024-09-21T23:11:30.779 INFO:teuthology.orchestra.run.smithi150.stdout:4607 punch from 0xa047329 to 0xa04baa8, (0x477f bytes) 2024-09-21T23:11:30.790 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:30.790 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:30.801 INFO:teuthology.orchestra.run.smithi150.stdout:4610 write 0xeb1fd5e thru 0xeb2ee5a (0xf0fd bytes) 2024-09-21T23:11:30.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:30.823 INFO:teuthology.orchestra.run.smithi150.stdout:4611 trunc from 0xeb2ee5b to 0x831e9ed 2024-09-21T23:11:30.848 INFO:teuthology.orchestra.run.smithi150.stdout:4612 trunc from 0x831e9ed to 0x8e7f108 2024-09-21T23:11:30.860 INFO:teuthology.orchestra.run.smithi150.stdout:4613 punch from 0x39cd81c to 0x39d2dc8, (0x55ac bytes) 2024-09-21T23:11:30.860 INFO:teuthology.orchestra.run.smithi150.stdout:4614 punch from 0x4d3c9e4 to 0x4d4a711, (0xdd2d bytes) 2024-09-21T23:11:30.860 INFO:teuthology.orchestra.run.smithi150.stdout:4615 punch from 0x7debe0a to 0x7df8463, (0xc659 bytes) 2024-09-21T23:11:30.861 INFO:teuthology.orchestra.run.smithi150.stdout:4616 write 0x2395652 thru 0x239991b (0x42ca bytes) 2024-09-21T23:11:30.864 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:30.859+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:30.870 INFO:teuthology.orchestra.run.smithi150.stdout:4617 trunc from 0x8e7f108 to 0xa446965 2024-09-21T23:11:30.883 INFO:teuthology.orchestra.run.smithi150.stdout:4618 punch from 0x83c72c5 to 0x83d6778, (0xf4b3 bytes) 2024-09-21T23:11:30.883 INFO:teuthology.orchestra.run.smithi150.stdout:4619 read 0x5bada89 thru 0x5bb0fc2 (0x353a bytes) 2024-09-21T23:11:30.908 INFO:teuthology.orchestra.run.smithi150.stdout:4620 write 0xe3c6c83 thru 0xe3d5044 (0xe3c2 bytes) 2024-09-21T23:11:30.915 INFO:teuthology.orchestra.run.smithi150.stdout:4622 write 0xd878c25 thru 0xd87eced (0x60c9 bytes) 2024-09-21T23:11:30.922 INFO:teuthology.orchestra.run.smithi150.stdout:4623 write 0x71448a4 thru 0x714f24f (0xa9ac bytes) 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108539643, 2024-09-21T23:11:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:11:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:11:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108468635000001, 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108404344, 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108373844999999, 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:11:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:11:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:11:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:11:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108225859000001, 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108169354000001, 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:11:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108114779000001, 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108059561000001, 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.108004932, 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:11:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:11:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107885782, 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:11:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:11:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107808347999999, 2024-09-21T23:11:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:11:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:11:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107742726000001, 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:11:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:11:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107678398000001, 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:11:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107617309999998, 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.993 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107556845000001, 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:11:30.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:30.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:30.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107492647000001, 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:30.998 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:30.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:31.002 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107418638999999, 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:11:31.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:11:31.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:11:31.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107381213, 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.007 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:11:31.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:11:31.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:31.011 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107334690999998, 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:31.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:11:31.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.015 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 18.107275196, 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.016 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:11:31.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:11:31.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:11:31.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:11:31.020 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.020 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.020 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.020 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.020 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.020 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:31.022 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:4624 trunc from 0xe3d5045 to 0x716b8d0 2024-09-21T23:11:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:4626 read 0x446c7a3 thru 0x4474fe4 (0x8842 bytes) 2024-09-21T23:11:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:4627 punch from 0x41cd87c to 0x41cf8e9, (0x206d bytes) 2024-09-21T23:11:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:4631 trunc from 0x716b8d0 to 0x3f951d1 2024-09-21T23:11:31.022 INFO:teuthology.orchestra.run.smithi150.stdout:4632 write 0x1738cac thru 0x1739932 (0xc87 bytes) 2024-09-21T23:11:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:4633 read 0x1131f9 thru 0x11c45b (0x9263 bytes) 2024-09-21T23:11:31.023 INFO:teuthology.orchestra.run.smithi150.stdout:4635 write 0xdea88b thru 0xdf5cad (0xb423 bytes) 2024-09-21T23:11:31.023 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:31.019+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:4637 writesame 0xce4e94a thru 0xce582d3 data_size 0x3(0x998a bytes) 2024-09-21T23:11:31.053 INFO:teuthology.orchestra.run.smithi150.stdout:4638 read 0x1c2215d thru 0x1c2c018 (0x9ebc bytes) 2024-09-21T23:11:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:4639 writesame 0x45f9c02 thru 0x4604c48 data_size 0x1(0xb047 bytes) 2024-09-21T23:11:31.097 INFO:teuthology.orchestra.run.smithi150.stdout:4641 read 0x8449868 thru 0x844c8ee (0x3087 bytes) 2024-09-21T23:11:31.097 INFO:teuthology.orchestra.run.smithi150.stdout:4642 read 0xb7eea7d thru 0xb7fb1a9 (0xc72d bytes) 2024-09-21T23:11:31.097 INFO:teuthology.orchestra.run.smithi150.stdout:4644 read 0xb5e7b20 thru 0xb5ea275 (0x2756 bytes) 2024-09-21T23:11:31.097 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:31.097 INFO:teuthology.orchestra.run.smithi150.stdout:4646 write 0x727541d thru 0x7284da5 (0xf989 bytes) 2024-09-21T23:11:31.105 INFO:teuthology.orchestra.run.smithi150.stdout:4649 write 0x95f48a3 thru 0x95fd44c (0x8baa bytes) 2024-09-21T23:11:31.111 INFO:teuthology.orchestra.run.smithi150.stdout:4650 punch from 0x9b8dc76 to 0x9b9255e, (0x48e8 bytes) 2024-09-21T23:11:31.117 INFO:teuthology.orchestra.run.smithi150.stdout:4651 trunc from 0xce582d4 to 0xaf2ad7f 2024-09-21T23:11:31.122 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:31.119+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:31.131 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:31.131 INFO:teuthology.orchestra.run.smithi150.stdout:4653 write 0x82aaef0 thru 0x82b1d4b (0x6e5c bytes) 2024-09-21T23:11:31.140 INFO:teuthology.orchestra.run.smithi150.stdout:4654 writesame 0x5bd7f16 thru 0x5bdb9a6 data_size 0x1d(0x3a91 bytes) 2024-09-21T23:11:31.147 INFO:teuthology.orchestra.run.smithi150.stdout:4655 clone 43 order 24 su 4194304 sc 7 2024-09-21T23:11:31.157 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:31.157 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:31.157 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:31.157 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:31.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:31.222 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:31.218+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:31.293 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:31.293 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:31.293 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:31.293 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:31.293 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:31.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:31.323 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:31.319+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:31.424 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:31.422+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:31.477 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:31.477 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:31.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:31.477 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.747787938999998, 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.478 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:31.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:11:31.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:11:31.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:11:31.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:11:31.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1004744459999998, 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1004417709999998, 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.488 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:11:31.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:11:31.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.100418436, 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.493 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.497 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1003858710000003, 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1002982489999997, 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:11:31.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1001795019999996, 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:11:31.506 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:11:31.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.510 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1001547370000004, 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1001335900000004, 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.515 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:11:31.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.519 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1000758929999996, 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:31.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:11:31.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:11:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1000410120000002, 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.1000154870000003, 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0999443010000007, 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:11:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0999084549999996, 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0998913619999993, 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:11:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:11:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:11:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0998753509999997, 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:11:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:11:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:11:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:11:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:11:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0998181030000005, 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:11:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:11:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0997367380000007, 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0997051879999997, 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.0995582909999992, 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:31.568 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:31.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:31.569 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:31.566+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:31.670 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:31.667+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:31.745 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:31.745 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:31.745 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:31.745 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:31.759 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:31.808 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:31.806+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:31.859 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone42 from 0xaf2ad7f (overlap 0x1053d39) to 0x0 2024-09-21T23:11:31.871 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:31.867+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:31.890 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:31.890 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:31.890 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:31.890 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:31.890 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:31.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:31.972 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:31.967+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:32.072 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:32.070+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:32.080 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.894542239000003, 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:11:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:11:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.894439152999993, 2024-09-21T23:11:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:11:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:11:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.894371804000002, 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.819184867999994, 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.819129099999998, 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.819097561000007, 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:11:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.819064811999993, 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:11:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.819037918999996, 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.819012646000004, 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:11:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:11:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.818981519000005, 2024-09-21T23:11:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.818338162999993, 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:11:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813152235999993, 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813130752999996, 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:11:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:11:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813106466999997, 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:11:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813088800000003, 2024-09-21T23:11:32.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:11:32.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:11:32.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813065214999995, 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:11:32.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:11:32.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813049199999995, 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813035369000005, 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.152 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:32.156 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.813004007999993, 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.157 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:32.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:32.161 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 84.812935585999995, 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:32.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:11:32.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.166 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.166 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:32.166 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:32.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:32.212 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:32.211+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:32.225 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-21T23:11:32.255 INFO:teuthology.orchestra.run.smithi150.stdout:4656 writesame 0x2f14a3c thru 0x2f2125b data_size 0x10(0xc820 bytes) 2024-09-21T23:11:32.272 INFO:teuthology.orchestra.run.smithi150.stdout:4658 read 0x96f5e9d thru 0x96fc456 (0x65ba bytes) 2024-09-21T23:11:32.272 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:32.272 INFO:teuthology.orchestra.run.smithi150.stdout:4661 writesame 0x29f6cb5 thru 0x29f8acc data_size 0x24(0x1e18 bytes) 2024-09-21T23:11:32.274 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:32.270+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:32.279 INFO:teuthology.orchestra.run.smithi150.stdout:4662 read 0x53fa6d3 thru 0x53fdb0b (0x3439 bytes) 2024-09-21T23:11:32.289 INFO:teuthology.orchestra.run.smithi150.stdout:4663 writesame 0xbbb855e thru 0xbbbbd39 data_size 0xb(0x37dc bytes) 2024-09-21T23:11:32.313 INFO:teuthology.orchestra.run.smithi150.stdout:4664 trunc from 0xbbbbd3a to 0x148fde6 2024-09-21T23:11:32.337 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:32.337 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:32.349 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:32.349 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:32.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.349 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.4425.0:86559 3.11 e96/83 3:8db23f99:::rbd_header.114922285636:head v 96'434, mlcod=96'432)", 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:32.349475+0000", 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00061083800000000005, 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062937599999999996, 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349475+0000", 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349475+0000", 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349477+0000", 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.136e-06 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349486+0000", 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5589999999999994e-06 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349505+0000", 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8406e-05 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349514+0000", 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2110000000000004e-06 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349594+0000", 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0222000000000005e-05 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:32.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:32.349627+0000", 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3331000000000003e-05 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:11:32.353 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:32.355 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:32.355 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:32.359 INFO:teuthology.orchestra.run.smithi150.stdout:4665 write 0x1810dc8 thru 0x1818ab7 (0x7cf0 bytes) 2024-09-21T23:11:32.363 INFO:teuthology.orchestra.run.smithi150.stdout:4666 read 0x127625a thru 0x127c9e3 (0x678a bytes) 2024-09-21T23:11:32.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:32.370 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:32.370 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:32.370 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-21T23:11:32.370 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:11:32.370 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:11:32.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-21T23:11:32.372 INFO:teuthology.orchestra.run.smithi150.stdout:4667 trunc from 0x1818ab8 to 0x6fd0275 2024-09-21T23:11:32.375 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:32.371+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:32.378 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:32.378 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:32.389 INFO:teuthology.orchestra.run.smithi150.stdout:4669 trunc from 0x6fd0275 to 0x17fdf7f 2024-09-21T23:11:32.402 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:32.402 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:32.422 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:32.422 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:32.424 INFO:teuthology.orchestra.run.smithi150.stdout:4670 writesame 0xa397304 thru 0xa39bca3 data_size 0x3e(0x49a0 bytes) 2024-09-21T23:11:32.424 INFO:teuthology.orchestra.run.smithi150.stdout:4671 punch from 0xd56b74 to 0xd5b071, (0x44fd bytes) 2024-09-21T23:11:32.424 INFO:teuthology.orchestra.run.smithi150.stdout:4673 punch from 0x3690d0c to 0x369e9c4, (0xdcb8 bytes) 2024-09-21T23:11:32.424 INFO:teuthology.orchestra.run.smithi150.stdout:4675 read 0x69ac705 thru 0x69b8305 (0xbc01 bytes) 2024-09-21T23:11:32.431 INFO:teuthology.orchestra.run.smithi150.stdout:4677 writesame 0xaa31b70 thru 0xaa3df52 data_size 0x1(0xc3e3 bytes) 2024-09-21T23:11:32.431 INFO:teuthology.orchestra.run.smithi150.stdout:4679 write 0x64a9030 thru 0x64b648c (0xd45d bytes) 2024-09-21T23:11:32.433 INFO:teuthology.orchestra.run.smithi150.stdout:4680 write 0x35d28d6 thru 0x35d4635 (0x1d60 bytes) 2024-09-21T23:11:32.436 INFO:teuthology.orchestra.run.smithi150.stdout:4681 write 0x8d14882 thru 0x8d1eac3 (0xa242 bytes) 2024-09-21T23:11:32.440 INFO:teuthology.orchestra.run.smithi150.stdout:4683 read 0x61e0432 thru 0x61ea612 (0xa1e1 bytes) 2024-09-21T23:11:32.440 INFO:teuthology.orchestra.run.smithi150.stdout:4684 read 0x8da0e5c thru 0x8da8386 (0x752b bytes) 2024-09-21T23:11:32.441 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:32.442 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:32.442 INFO:teuthology.orchestra.run.smithi150.stdout:4685 read 0x97b20b7 thru 0x97be665 (0xc5af bytes) 2024-09-21T23:11:32.442 INFO:teuthology.orchestra.run.smithi150.stdout:4687 clone 44 order 25 su 2097152 sc 5 2024-09-21T23:11:32.459 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:32.459 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:32.476 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:32.476 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:32.477 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:32.474+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:32.528 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:32.529 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "" 2024-09-21T23:11:32.529 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:32.532 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:32.532 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:32.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:32.577 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:32.575+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:32.631 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:11:32.681 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:32.679+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:32.719 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:32.720 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:32.734 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:32.784 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:32.783+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:32.815 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone43 from 0xaa3df53 (overlap 0x148fde6) to 0x62b291 2024-09-21T23:11:32.863 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:32.863 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:32.878 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:32.879 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:32.878+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:32.981 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:32.979+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:33.008 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-21T23:11:33.010 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:4690 write 0x445bf0b thru 0x446bb8e (0xfc84 bytes) 2024-09-21T23:11:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:4693 read 0x4d3f09e thru 0x4d4d419 (0xe37c bytes) 2024-09-21T23:11:33.029 INFO:teuthology.orchestra.run.smithi150.stdout:4695 read 0x66a4964 thru 0x66a9b3f (0x51dc bytes) 2024-09-21T23:11:33.029 INFO:teuthology.orchestra.run.smithi150.stdout:4696 punch from 0x5ff382a to 0x5ff6ed5, (0x36ab bytes) 2024-09-21T23:11:33.029 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:33.029 INFO:teuthology.orchestra.run.smithi150.stdout:4701 write 0x66a92c5 thru 0x66a97ad (0x4e9 bytes) 2024-09-21T23:11:33.052 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:33.052 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:33.056 INFO:teuthology.orchestra.run.smithi150.stdout:4704 writesame 0x82aaa85 thru 0x82b81af data_size 0x3(0xd72b bytes) 2024-09-21T23:11:33.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:33.082 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:33.078+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:33.093 INFO:teuthology.orchestra.run.smithi150.stdout:4705 punch from 0x8e207cb to 0x8e25bfa, (0x542f bytes) 2024-09-21T23:11:33.093 INFO:teuthology.orchestra.run.smithi150.stdout:4706 write 0x97d4378 thru 0x97d94fa (0x5183 bytes) 2024-09-21T23:11:33.096 INFO:teuthology.orchestra.run.smithi150.stdout:4707 writesame 0x351da8 thru 0x35363e data_size 0x1(0x1897 bytes) 2024-09-21T23:11:33.104 INFO:teuthology.orchestra.run.smithi150.stdout:4708 writesame 0x2dd324d thru 0x2dd9fcb data_size 0x1(0x6d7f bytes) 2024-09-21T23:11:33.151 INFO:teuthology.orchestra.run.smithi150.stdout:4709 read 0x96e5ac5 thru 0x96eb0ff (0x563b bytes) 2024-09-21T23:11:33.152 INFO:teuthology.orchestra.run.smithi150.stdout:4710 write 0x4ee9521 thru 0x4ef2ef6 (0x99d6 bytes) 2024-09-21T23:11:33.156 INFO:teuthology.orchestra.run.smithi150.stdout:4711 read 0x12c092e thru 0x12c8ad7 (0x81aa bytes) 2024-09-21T23:11:33.157 INFO:teuthology.orchestra.run.smithi150.stdout:4714 read 0x83a0100 thru 0x83a4f13 (0x4e14 bytes) 2024-09-21T23:11:33.158 INFO:teuthology.orchestra.run.smithi150.stdout:4715 read 0x7dbc520 thru 0x7dbdb14 (0x15f5 bytes) 2024-09-21T23:11:33.162 INFO:teuthology.orchestra.run.smithi150.stdout:4716 trunc from 0xaa3df53 to 0xcf205e6 2024-09-21T23:11:33.171 INFO:teuthology.orchestra.run.smithi150.stdout:4718 writesame 0x2363662 thru 0x2369e43 data_size 0x1(0x67e2 bytes) 2024-09-21T23:11:33.183 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:33.178+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:33.206 INFO:teuthology.orchestra.run.smithi150.stdout:4719 writesame 0x65a30c7 thru 0x65a88d9 data_size 0x1(0x5813 bytes) 2024-09-21T23:11:33.219 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:33.219 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:33.227 INFO:teuthology.orchestra.run.smithi150.stdout:4721 read 0x7ff1f3c thru 0x7ff239d (0x462 bytes) 2024-09-21T23:11:33.227 INFO:teuthology.orchestra.run.smithi150.stdout:4722 read 0x9f1f377 thru 0x9f2e92a (0xf5b4 bytes) 2024-09-21T23:11:33.242 INFO:teuthology.orchestra.run.smithi150.stdout:4723 trunc from 0xcf205e6 to 0x49efd88 2024-09-21T23:11:33.247 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:4725 writesame 0x578ebee thru 0x5790f89 data_size 0x35(0x239c bytes) 2024-09-21T23:11:33.284 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:33.282+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:33.296 INFO:teuthology.orchestra.run.smithi150.stdout:4728 write 0xba21ff8 thru 0xba2402e (0x2037 bytes) 2024-09-21T23:11:33.303 INFO:teuthology.orchestra.run.smithi150.stdout:4729 punch from 0x30f8d2f to 0x31007ce, (0x7a9f bytes) 2024-09-21T23:11:33.303 INFO:teuthology.orchestra.run.smithi150.stdout:4730 clone 45 order 24 su 262144 sc 15 2024-09-21T23:11:33.385 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:33.383+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:33.400 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:33.400 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:33.414 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:33.486 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:33.482+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:33.574 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:33.574 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:33.586 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:33.583+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:33.589 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:33.621 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:11:33.688 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:33.687+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:33.744 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:33.744 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:33.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:33.800 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:33.799+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:33.868 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone44 from 0xba2402f (overlap 0x49efd88) to 0x3b2646c 2024-09-21T23:11:33.889 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:33.886+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:33.928 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:33.928 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:33.944 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:33.990 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:33.986+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:34.072 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:34.072 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:34.086 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:34.133 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:34.131+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:34.192 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:34.187+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:34.219 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:34.220 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:34.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:34.293 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:34.289+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:34.302 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-21T23:11:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:4734 write 0x1d8bae9 thru 0x1d93c2e (0x8146 bytes) 2024-09-21T23:11:34.367 INFO:teuthology.orchestra.run.smithi150.stdout:4736 write 0xdf9ba93 thru 0xdfa4c04 (0x9172 bytes) 2024-09-21T23:11:34.394 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:34.391+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:34.394 INFO:teuthology.orchestra.run.smithi150.stdout:4737 trunc from 0xdfa4c05 to 0x7df034b 2024-09-21T23:11:34.432 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:34.432 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:34.451 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:34.487 INFO:teuthology.orchestra.run.smithi150.stdout:4739 writesame 0xb9b2a27 thru 0xb9bb638 data_size 0x1(0x8c12 bytes) 2024-09-21T23:11:34.489 INFO:teuthology.orchestra.run.smithi150.stdout:4741 write 0x796f977 thru 0x797baaa (0xc134 bytes) 2024-09-21T23:11:34.491 INFO:teuthology.orchestra.run.smithi150.stdout:4742 writesame 0x9611ee0 thru 0x961453d data_size 0x3(0x265e bytes) 2024-09-21T23:11:34.493 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:34.494 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:34.489+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:34.507 INFO:teuthology.orchestra.run.smithi150.stdout:4743 trunc from 0xb9bb639 to 0x37757f2 2024-09-21T23:11:34.545 INFO:teuthology.orchestra.run.smithi150.stdout:4744 write 0xd55e455 thru 0xd55eea8 (0xa54 bytes) 2024-09-21T23:11:34.547 INFO:teuthology.orchestra.run.smithi150.stdout:4745 read 0x224857d thru 0x2249b94 (0x1618 bytes) 2024-09-21T23:11:34.548 INFO:teuthology.orchestra.run.smithi150.stdout:4747 punch from 0x8b0310e to 0x8b07005, (0x3ef7 bytes) 2024-09-21T23:11:34.548 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.550 INFO:teuthology.orchestra.run.smithi150.stdout:4750 trunc from 0xd55eea9 to 0xbcbd162 2024-09-21T23:11:34.574 INFO:teuthology.orchestra.run.smithi150.stdout:4751 write 0xdcc72ff thru 0xdcc9a2b (0x272d bytes) 2024-09-21T23:11:34.576 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.576 INFO:teuthology.orchestra.run.smithi150.stdout:4754 read 0x27db17c thru 0x27df591 (0x4416 bytes) 2024-09-21T23:11:34.577 INFO:teuthology.orchestra.run.smithi150.stdout:4755 writesame 0x6591f3b thru 0x65924f9 data_size 0x1(0x5bf bytes) 2024-09-21T23:11:34.580 INFO:teuthology.orchestra.run.smithi150.stdout:4756 write 0x4d04df5 thru 0x4d05441 (0x64d bytes) 2024-09-21T23:11:34.581 INFO:teuthology.orchestra.run.smithi150.stdout:4757 read 0xd703c80 thru 0xd70ab28 (0x6ea9 bytes) 2024-09-21T23:11:34.582 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.582 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.582 INFO:teuthology.orchestra.run.smithi150.stdout:4760 read 0xc43a5b0 thru 0xc443b95 (0x95e6 bytes) 2024-09-21T23:11:34.582 INFO:teuthology.orchestra.run.smithi150.stdout:4761 writesame 0xb270cff thru 0xb273d5a data_size 0xa(0x305c bytes) 2024-09-21T23:11:34.584 INFO:teuthology.orchestra.run.smithi150.stdout:4762 write 0xad69b41 thru 0xad770ef (0xd5af bytes) 2024-09-21T23:11:34.587 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.587 INFO:teuthology.orchestra.run.smithi150.stdout:4765 write 0x6aeeef2 thru 0x6af1a90 (0x2b9f bytes) 2024-09-21T23:11:34.594 INFO:teuthology.orchestra.run.smithi150.stdout:4766 trunc from 0xdcc9a2c to 0xb7ca4ff 2024-09-21T23:11:34.595 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:34.589+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:34.622 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:34.622 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:34.626 INFO:teuthology.orchestra.run.smithi150.stdout:4767 trunc from 0xb7ca4ff to 0x23b89e 2024-09-21T23:11:34.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:34.657 INFO:teuthology.orchestra.run.smithi150.stdout:4768 read 0x154ab6 thru 0x15adaa (0x62f5 bytes) 2024-09-21T23:11:34.657 INFO:teuthology.orchestra.run.smithi150.stdout:4770 read 0x184d41 thru 0x188ab7 (0x3d77 bytes) 2024-09-21T23:11:34.659 INFO:teuthology.orchestra.run.smithi150.stdout:4771 punch from 0x5e534 to 0x66ea4, (0x8970 bytes) 2024-09-21T23:11:34.667 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:34.667 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:34.681 INFO:teuthology.orchestra.run.smithi150.stdout:4773 writesame 0xa5b603f thru 0xa5bccc6 data_size 0x2e(0x6c88 bytes) 2024-09-21T23:11:34.683 INFO:teuthology.orchestra.run.smithi150.stdout:4774 read 0x51b809f thru 0x51c4879 (0xc7db bytes) 2024-09-21T23:11:34.683 INFO:teuthology.orchestra.run.smithi150.stdout:4775 write 0xdef400 thru 0xdfdef0 (0xeaf1 bytes) 2024-09-21T23:11:34.685 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:34.686 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:34.696 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:34.693+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:34.697 INFO:teuthology.orchestra.run.smithi150.stdout:4777 writesame 0xb18bad0 thru 0xb1991d4 data_size 0x65(0xd705 bytes) 2024-09-21T23:11:34.699 INFO:teuthology.orchestra.run.smithi150.stdout:4778 write 0x7572600 thru 0x75772ee (0x4cef bytes) 2024-09-21T23:11:34.701 INFO:teuthology.orchestra.run.smithi150.stdout:4779 punch from 0x8ee747b to 0x8eed7d8, (0x635d bytes) 2024-09-21T23:11:34.701 INFO:teuthology.orchestra.run.smithi150.stdout:4781 trunc from 0xb1991d5 to 0xb16e926 2024-09-21T23:11:34.704 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:34.704 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:34.711 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.711 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.725 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:34.725 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:34.726 INFO:teuthology.orchestra.run.smithi150.stdout:4785 write 0xec176df thru 0xec214be (0x9de0 bytes) 2024-09-21T23:11:34.732 INFO:teuthology.orchestra.run.smithi150.stdout:4786 write 0x9c9429c thru 0x9c9955e (0x52c3 bytes) 2024-09-21T23:11:34.738 INFO:teuthology.orchestra.run.smithi150.stdout:4787 writesame 0x84562f7 thru 0x845f52e data_size 0x4(0x9238 bytes) 2024-09-21T23:11:34.743 INFO:teuthology.orchestra.run.smithi150.stdout:4788 punch from 0xfc31df to 0xfcba36, (0x8857 bytes) 2024-09-21T23:11:34.743 INFO:teuthology.orchestra.run.smithi150.stdout:4791 write 0x524d4ba thru 0x5257e68 (0xa9af bytes) 2024-09-21T23:11:34.746 INFO:teuthology.orchestra.run.smithi150.stdout:4792 punch from 0x2aa9d93 to 0x2ab0d08, (0x6f75 bytes) 2024-09-21T23:11:34.746 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.746 INFO:teuthology.orchestra.run.smithi150.stdout:4794 write 0x4e0eab2 thru 0x4e18e0e (0xa35d bytes) 2024-09-21T23:11:34.747 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:34.748 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:34.750 INFO:teuthology.orchestra.run.smithi150.stdout:4796 write 0x4b0c9a thru 0x4b5ab2 (0x4e19 bytes) 2024-09-21T23:11:34.753 INFO:teuthology.orchestra.run.smithi150.stdout:4797 writesame 0x6a5fb84 thru 0x6a60913 data_size 0x1f(0xd90 bytes) 2024-09-21T23:11:34.756 INFO:teuthology.orchestra.run.smithi150.stdout:4798 read 0xdf0da3e thru 0xdf10907 (0x2eca bytes) 2024-09-21T23:11:34.756 INFO:teuthology.orchestra.run.smithi150.stdout:4799 write 0x4d1309b thru 0x4d17a89 (0x49ef bytes) 2024-09-21T23:11:34.758 INFO:teuthology.orchestra.run.smithi150.stdout:4800 writesame 0x3fba0c2 thru 0x3fbd881 data_size 0x20(0x37c0 bytes) 2024-09-21T23:11:34.761 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.761 INFO:teuthology.orchestra.run.smithi150.stdout:4802 read 0x1dd2dd thru 0x1e1df0 (0x4b14 bytes) 2024-09-21T23:11:34.763 INFO:teuthology.orchestra.run.smithi150.stdout:4803 write 0x887fd16 thru 0x888216b (0x2456 bytes) 2024-09-21T23:11:34.766 INFO:teuthology.orchestra.run.smithi150.stdout:4804 write 0x8ed4f8 thru 0x8eeba2 (0x16ab bytes) 2024-09-21T23:11:34.769 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.769 INFO:teuthology.orchestra.run.smithi150.stdout:4807 read 0x439834 thru 0x43bcc4 (0x2491 bytes) 2024-09-21T23:11:34.769 INFO:teuthology.orchestra.run.smithi150.stdout:4809 punch from 0xa8ac32c to 0xa8ba2b9, (0xdf8d bytes) 2024-09-21T23:11:34.769 INFO:teuthology.orchestra.run.smithi150.stdout:4810 write 0x4741bd8 thru 0x4742451 (0x87a bytes) 2024-09-21T23:11:34.770 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:34.770 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:34.771 INFO:teuthology.orchestra.run.smithi150.stdout:4811 write 0x557c8d3 thru 0x5581b67 (0x5295 bytes) 2024-09-21T23:11:34.773 INFO:teuthology.orchestra.run.smithi150.stdout:4813 write 0xc35334b thru 0xc3568b1 (0x3567 bytes) 2024-09-21T23:11:34.776 INFO:teuthology.orchestra.run.smithi150.stdout:4815 write 0xc64f352 thru 0xc657579 (0x8228 bytes) 2024-09-21T23:11:34.780 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.780 INFO:teuthology.orchestra.run.smithi150.stdout:4817 read 0x5dc4682 thru 0x5dc9e46 (0x57c5 bytes) 2024-09-21T23:11:34.780 INFO:teuthology.orchestra.run.smithi150.stdout:4818 read 0x3469c13 thru 0x346b9ac (0x1d9a bytes) 2024-09-21T23:11:34.781 INFO:teuthology.orchestra.run.smithi150.stdout:4820 writesame 0x392ce27 thru 0x3938f74 data_size 0x6d(0xc14e bytes) 2024-09-21T23:11:34.784 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.784 INFO:teuthology.orchestra.run.smithi150.stdout:4823 write 0x32e2939 thru 0x32f1dba (0xf482 bytes) 2024-09-21T23:11:34.786 INFO:teuthology.orchestra.run.smithi150.stdout:4824 punch from 0xa8d22ec to 0xa8d2819, (0x52d bytes) 2024-09-21T23:11:34.786 INFO:teuthology.orchestra.run.smithi150.stdout:4826 read 0x99b1033 thru 0x99b81a5 (0x7173 bytes) 2024-09-21T23:11:34.787 INFO:teuthology.orchestra.run.smithi150.stdout:4827 read 0xa2be726 thru 0xa2c0d46 (0x2621 bytes) 2024-09-21T23:11:34.788 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:34.788 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 83.712859593000005, 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.457308098, 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.457283878, 2024-09-21T23:11:34.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.457237492000001, 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.457222269000001, 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:11:34.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:11:34.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77015 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.446103473000001, 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059342726999999998, 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.811 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77015 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341443+0000", 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341445+0000", 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341453+0000", 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4869999999999996e-06 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341458+0000", 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400574+0000", 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059115684000000002 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400680+0000", 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010637399999999999 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400785+0000", 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001047 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.815 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.446055514999999, 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:34.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:11:34.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:11:34.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.446032788, 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:11:34.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:11:34.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.446004191, 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.445991986999999, 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.831 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.445972996, 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.832 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:11:34.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:11:34.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:11:34.835 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.44563868, 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.836 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.445594553999999, 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.840 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:11:34.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:11:34.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.445570597, 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.445530394, 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:11:34.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:11:34.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.434391204000001, 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.852 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.43436925, 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.856 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.434342832, 2024-09-21T23:11:34.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:34.864 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.434316536000001, 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:11:34.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.434293924, 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:34.873 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:34.873 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:34.873 INFO:teuthology.orchestra.run.smithi150.stdout:4828 writesame 0xd05c877 thru 0xd0643c4 data_size 0x3(0x7b4e bytes) 2024-09-21T23:11:34.873 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:34.873 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:34.874 INFO:teuthology.orchestra.run.smithi150.stdout:4830 trunc from 0xec214bf to 0x897d1c0 2024-09-21T23:11:34.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:34.875 INFO:teuthology.orchestra.run.smithi150.stdout:4831 write 0x560a1b9 thru 0x560f021 (0x4e69 bytes) 2024-09-21T23:11:34.875 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:34.875 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:34.876 INFO:teuthology.orchestra.run.smithi150.stdout:4832 writesame 0xdbbb198 thru 0xdbbee90 data_size 0x2b(0x3cf9 bytes) 2024-09-21T23:11:34.876 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.876 INFO:teuthology.orchestra.run.smithi150.stdout:4834 read 0x31e2b75 thru 0x31ec620 (0x9aac bytes) 2024-09-21T23:11:34.876 INFO:teuthology.orchestra.run.smithi150.stdout:4835 read 0x5c228f3 thru 0x5c31214 (0xe922 bytes) 2024-09-21T23:11:34.876 INFO:teuthology.orchestra.run.smithi150.stdout:4836 punch from 0x1b2ec78 to 0x1b2f19f, (0x527 bytes) 2024-09-21T23:11:34.877 INFO:teuthology.orchestra.run.smithi150.stdout:4837 trunc from 0xdbbee91 to 0xb22477 2024-09-21T23:11:34.878 INFO:teuthology.orchestra.run.smithi150.stdout:4838 read 0x70f972 thru 0x714167 (0x47f6 bytes) 2024-09-21T23:11:34.878 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.878 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:34.873+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:34.896 INFO:teuthology.orchestra.run.smithi150.stdout:4840 writesame 0xc247445 thru 0xc25145c data_size 0x2f(0xa018 bytes) 2024-09-21T23:11:34.913 INFO:teuthology.orchestra.run.smithi150.stdout:4841 trunc from 0xc25145d to 0xd71a3c 2024-09-21T23:11:34.923 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.946 INFO:teuthology.orchestra.run.smithi150.stdout:4844 write 0xb9b9fa4 thru 0xb9bba18 (0x1a75 bytes) 2024-09-21T23:11:34.948 INFO:teuthology.orchestra.run.smithi150.stdout:4845 punch from 0x48e31b8 to 0x48ed060, (0x9ea8 bytes) 2024-09-21T23:11:34.948 INFO:teuthology.orchestra.run.smithi150.stdout:4846 writesame 0x1eb945b thru 0x1ec8465 data_size 0x1d(0xf00b bytes) 2024-09-21T23:11:34.951 INFO:teuthology.orchestra.run.smithi150.stdout:4847 write 0x1b92c4d thru 0x1b96767 (0x3b1b bytes) 2024-09-21T23:11:34.953 INFO:teuthology.orchestra.run.smithi150.stdout:4848 read 0x593a100 thru 0x59433a5 (0x92a6 bytes) 2024-09-21T23:11:34.953 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:34.953 INFO:teuthology.orchestra.run.smithi150.stdout:4852 read 0xa83b569 thru 0xa843d6c (0x8804 bytes) 2024-09-21T23:11:34.954 INFO:teuthology.orchestra.run.smithi150.stdout:4853 write 0x24ab262 thru 0x24ac2b9 (0x1058 bytes) 2024-09-21T23:11:34.955 INFO:teuthology.orchestra.run.smithi150.stdout:4854 write 0x3c7c775 thru 0x3c81e33 (0x56bf bytes) 2024-09-21T23:11:34.957 INFO:teuthology.orchestra.run.smithi150.stdout:4855 write 0xb5dd3c8 thru 0xb5ece3f (0xfa78 bytes) 2024-09-21T23:11:34.959 INFO:teuthology.orchestra.run.smithi150.stdout:4857 read 0x9c52b4d thru 0x9c5eb4c (0xc000 bytes) 2024-09-21T23:11:34.959 INFO:teuthology.orchestra.run.smithi150.stdout:4858 write 0x3861c2e thru 0x38625ff (0x9d2 bytes) 2024-09-21T23:11:34.961 INFO:teuthology.orchestra.run.smithi150.stdout:4859 write 0x6a300c0 thru 0x6a31712 (0x1653 bytes) 2024-09-21T23:11:34.963 INFO:teuthology.orchestra.run.smithi150.stdout:4860 read 0x195ec37 thru 0x1967956 (0x8d20 bytes) 2024-09-21T23:11:34.963 INFO:teuthology.orchestra.run.smithi150.stdout:4861 punch from 0x7ecb81b to 0x7ed0b86, (0x536b bytes) 2024-09-21T23:11:34.964 INFO:teuthology.orchestra.run.smithi150.stdout:4863 punch from 0xe36f93 to 0xe43978, (0xc9e5 bytes) 2024-09-21T23:11:34.966 INFO:teuthology.orchestra.run.smithi150.stdout:4864 trunc from 0xb9bba19 to 0x9c6efcd 2024-09-21T23:11:34.978 INFO:teuthology.orchestra.run.smithi150.stdout:4867 writesame 0x6965f51 thru 0x6973f8a data_size 0x1b(0xe03a bytes) 2024-09-21T23:11:34.978 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:34.975+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:34.980 INFO:teuthology.orchestra.run.smithi150.stdout:4868 read 0x3e5108e thru 0x3e56c36 (0x5ba9 bytes) 2024-09-21T23:11:34.985 INFO:teuthology.orchestra.run.smithi150.stdout:4869 trunc from 0x9c6efcd to 0x709bdaf 2024-09-21T23:11:35.014 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:35.014 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:35.014 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:35.015 INFO:teuthology.orchestra.run.smithi150.stdout:4870 write 0xdb6701a thru 0xdb73e7d (0xce64 bytes) 2024-09-21T23:11:35.018 INFO:teuthology.orchestra.run.smithi150.stdout:4871 write 0x7df7b31 thru 0x7e00c5d (0x912d bytes) 2024-09-21T23:11:35.023 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.023 INFO:teuthology.orchestra.run.smithi150.stdout:4873 read 0x3cf3112 thru 0x3d02eb7 (0xfda6 bytes) 2024-09-21T23:11:35.023 INFO:teuthology.orchestra.run.smithi150.stdout:4874 write 0x83a839e thru 0x83ad412 (0x5075 bytes) 2024-09-21T23:11:35.026 INFO:teuthology.orchestra.run.smithi150.stdout:4875 read 0xa1404fe thru 0xa1475ae (0x70b1 bytes) 2024-09-21T23:11:35.027 INFO:teuthology.orchestra.run.smithi150.stdout:4876 punch from 0x4b02170 to 0x4b0ca6b, (0xa8fb bytes) 2024-09-21T23:11:35.029 INFO:teuthology.orchestra.run.smithi150.stdout:4877 write 0xa0df005 thru 0xa0ed842 (0xe83e bytes) 2024-09-21T23:11:35.030 INFO:teuthology.orchestra.run.smithi150.stdout:4878 writesame 0x5f7b7c5 thru 0x5f820f7 data_size 0x2f(0x6933 bytes) 2024-09-21T23:11:35.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:35.031 INFO:teuthology.orchestra.run.smithi150.stdout:4881 write 0x35cd32 thru 0x35f766 (0x2a35 bytes) 2024-09-21T23:11:35.032 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.033 INFO:teuthology.orchestra.run.smithi150.stdout:4883 write 0x3dd0f55 thru 0x3dd46d5 (0x3781 bytes) 2024-09-21T23:11:35.042 INFO:teuthology.orchestra.run.smithi150.stdout:4885 writesame 0xe80384d thru 0xe80fe39 data_size 0x1(0xc5ed bytes) 2024-09-21T23:11:35.068 INFO:teuthology.orchestra.run.smithi150.stdout:4887 trunc from 0xe80fe3a to 0x709cea 2024-09-21T23:11:35.079 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:35.073+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:35.086 INFO:teuthology.orchestra.run.smithi150.stdout:4888 read 0x2b1e44 thru 0x2b9269 (0x7426 bytes) 2024-09-21T23:11:35.118 INFO:teuthology.orchestra.run.smithi150.stdout:4889 writesame 0xb975a69 thru 0xb977270 data_size 0x4(0x1808 bytes) 2024-09-21T23:11:35.120 INFO:teuthology.orchestra.run.smithi150.stdout:4890 read 0xa7a62e7 thru 0xa7a8f7c (0x2c96 bytes) 2024-09-21T23:11:35.140 INFO:teuthology.orchestra.run.smithi150.stdout:4891 trunc from 0xb977271 to 0x1c8d0e3 2024-09-21T23:11:35.155 INFO:teuthology.orchestra.run.smithi150.stdout:4892 read 0x781d25 thru 0x7880e8 (0x63c4 bytes) 2024-09-21T23:11:35.155 INFO:teuthology.orchestra.run.smithi150.stdout:4893 read 0x134bd68 thru 0x134d330 (0x15c9 bytes) 2024-09-21T23:11:35.156 INFO:teuthology.orchestra.run.smithi150.stdout:4895 punch from 0xecdc45 to 0xed4f1a, (0x72d5 bytes) 2024-09-21T23:11:35.156 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.156 INFO:teuthology.orchestra.run.smithi150.stdout:4897 punch from 0x9c1ff2 to 0x9c7db0, (0x5dbe bytes) 2024-09-21T23:11:35.156 INFO:teuthology.orchestra.run.smithi150.stdout:4898 read 0x20b28b thru 0x210d1c (0x5a92 bytes) 2024-09-21T23:11:35.157 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:35.157 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:35.157 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:35.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:35.181 INFO:teuthology.orchestra.run.smithi150.stdout:4902 writesame 0xb3efad0 thru 0xb3f886b data_size 0x39(0x8d9c bytes) 2024-09-21T23:11:35.183 INFO:teuthology.orchestra.run.smithi150.stdout:4903 write 0x9cecf3f thru 0x9cf44f0 (0x75b2 bytes) 2024-09-21T23:11:35.185 INFO:teuthology.orchestra.run.smithi150.stdout:4904 write 0x6a6424f thru 0x6a68396 (0x4148 bytes) 2024-09-21T23:11:35.187 INFO:teuthology.orchestra.run.smithi150.stdout:4905 trunc from 0xb3f886c to 0xb578350 2024-09-21T23:11:35.195 INFO:teuthology.orchestra.run.smithi150.stdout:4907 read 0x85fbfdc thru 0x8602c66 (0x6c8b bytes) 2024-09-21T23:11:35.195 INFO:teuthology.orchestra.run.smithi150.stdout:4908 write 0x2865e44 thru 0x286eca8 (0x8e65 bytes) 2024-09-21T23:11:35.197 INFO:teuthology.orchestra.run.smithi150.stdout:4909 read 0x74d3f0c thru 0x74df48e (0xb583 bytes) 2024-09-21T23:11:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.198 INFO:teuthology.orchestra.run.smithi150.stdout:4911 write 0x530dc5 thru 0x537487 (0x66c3 bytes) 2024-09-21T23:11:35.199 INFO:teuthology.orchestra.run.smithi150.stdout:4912 read 0x877680c thru 0x8785b99 (0xf38e bytes) 2024-09-21T23:11:35.200 INFO:teuthology.orchestra.run.smithi150.stdout:4913 read 0x6f01de2 thru 0x6f0206b (0x28a bytes) 2024-09-21T23:11:35.201 INFO:teuthology.orchestra.run.smithi150.stdout:4914 read 0x515868f thru 0x5158bb8 (0x52a bytes) 2024-09-21T23:11:35.213 INFO:teuthology.orchestra.run.smithi150.stdout:4915 write 0xd5433e3 thru 0xd543c38 (0x856 bytes) 2024-09-21T23:11:35.215 INFO:teuthology.orchestra.run.smithi150.stdout:4916 writesame 0xce0e8b thru 0xceca87 data_size 0x7d(0xbbfd bytes) 2024-09-21T23:11:35.217 INFO:teuthology.orchestra.run.smithi150.stdout:4917 punch from 0x47cb875 to 0x47cff3e, (0x46c9 bytes) 2024-09-21T23:11:35.218 INFO:teuthology.orchestra.run.smithi150.stdout:4918 write 0xd2568a thru 0xd350e8 (0xfa5f bytes) 2024-09-21T23:11:35.220 INFO:teuthology.orchestra.run.smithi150.stdout:4919 punch from 0xa5904ce to 0xa59daa6, (0xd5d8 bytes) 2024-09-21T23:11:35.220 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:35.217+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:35.239 INFO:teuthology.orchestra.run.smithi150.stdout:4920 trunc from 0xd543c39 to 0x1bc299c 2024-09-21T23:11:35.282 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:35.279+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:35.288 INFO:teuthology.orchestra.run.smithi150.stdout:4922 writesame 0xe085356 thru 0xe0854ac data_size 0x1(0x157 bytes) 2024-09-21T23:11:35.291 INFO:teuthology.orchestra.run.smithi150.stdout:4923 read 0xd755044 thru 0xd75f40e (0xa3cb bytes) 2024-09-21T23:11:35.291 INFO:teuthology.orchestra.run.smithi150.stdout:4924 punch from 0xb390365 to 0xb3936cf, (0x336a bytes) 2024-09-21T23:11:35.291 INFO:teuthology.orchestra.run.smithi150.stdout:4927 read 0x2fb1649 thru 0x2fb5c4f (0x4607 bytes) 2024-09-21T23:11:35.292 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.292 INFO:teuthology.orchestra.run.smithi150.stdout:4929 punch from 0x87b784a to 0x87c2333, (0xaae9 bytes) 2024-09-21T23:11:35.292 INFO:teuthology.orchestra.run.smithi150.stdout:4933 punch from 0x6c9aaaf to 0x6ca4803, (0x9d54 bytes) 2024-09-21T23:11:35.292 INFO:teuthology.orchestra.run.smithi150.stdout:4934 write 0x845017f thru 0x845a8ae (0xa730 bytes) 2024-09-21T23:11:35.295 INFO:teuthology.orchestra.run.smithi150.stdout:4935 read 0x945b441 thru 0x945e9a4 (0x3564 bytes) 2024-09-21T23:11:35.305 INFO:teuthology.orchestra.run.smithi150.stdout:4936 trunc from 0xe0854ad to 0x8a1fbc9 2024-09-21T23:11:35.339 INFO:teuthology.orchestra.run.smithi150.stdout:4938 write 0xae83358 thru 0xae90f39 (0xdbe2 bytes) 2024-09-21T23:11:35.342 INFO:teuthology.orchestra.run.smithi150.stdout:4939 punch from 0x9ae3514 to 0x9ae6c89, (0x3775 bytes) 2024-09-21T23:11:35.343 INFO:teuthology.orchestra.run.smithi150.stdout:4940 writesame 0xaa8eb19 thru 0xaa9d318 data_size 0x74(0xe800 bytes) 2024-09-21T23:11:35.345 INFO:teuthology.orchestra.run.smithi150.stdout:4941 read 0x3ebd6be thru 0x3ec9355 (0xbc98 bytes) 2024-09-21T23:11:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.346 INFO:teuthology.orchestra.run.smithi150.stdout:4943 read 0x39dc2fc thru 0x39e68ba (0xa5bf bytes) 2024-09-21T23:11:35.347 INFO:teuthology.orchestra.run.smithi150.stdout:4944 punch from 0x6aa809a to 0x6aafe0a, (0x7d70 bytes) 2024-09-21T23:11:35.347 INFO:teuthology.orchestra.run.smithi150.stdout:4945 write 0x60da299 thru 0x60dcf45 (0x2cad bytes) 2024-09-21T23:11:35.350 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.350 INFO:teuthology.orchestra.run.smithi150.stdout:4949 write 0x622fb18 thru 0x623830e (0x87f7 bytes) 2024-09-21T23:11:35.351 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:35.351 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:35.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:35.351 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:35.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.351 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.351 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.525399966999998, 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:11:35.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:11:35.355 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.525328958999999, 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.356 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:11:35.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.525264667999998, 2024-09-21T23:11:35.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:11:35.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:11:35.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.525234169000001, 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.365 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:11:35.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.525086182999999, 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.369 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:11:35.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.373 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.525029677999999, 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:11:35.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:11:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:11:35.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524975102999999, 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:11:35.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:11:35.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524919884999999, 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:11:35.382 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524865255999998, 2024-09-21T23:11:35.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.390 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524746105999998, 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524668672000001, 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.395 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:11:35.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:11:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:11:35.398 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.52460305, 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.399 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524538721999999, 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.403 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:11:35.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:11:35.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524477634, 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:11:35.407 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:11:35.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524417168999999, 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:11:35.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:11:35.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524352970999999, 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:11:35.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:35.420 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524278963, 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:11:35.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524241537000002, 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524195015, 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:11:35.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:11:35.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:11:35.432 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.524135520000002, 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.433 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:11:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:11:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:11:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:11:35.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.438 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.439 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.439 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:35.440 INFO:teuthology.orchestra.run.smithi150.stdout:4950 read 0x9e2772e thru 0x9e2f1f8 (0x7acb bytes) 2024-09-21T23:11:35.441 INFO:teuthology.orchestra.run.smithi150.stdout:4951 trunc from 0xae90f3a to 0x2c4a7d7 2024-09-21T23:11:35.441 INFO:teuthology.orchestra.run.smithi150.stdout:4952 writesame 0x33dc91f thru 0x33e7605 data_size 0x1(0xace7 bytes) 2024-09-21T23:11:35.441 INFO:teuthology.orchestra.run.smithi150.stdout:4953 write 0x380d084 thru 0x380f325 (0x22a2 bytes) 2024-09-21T23:11:35.441 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.441 INFO:teuthology.orchestra.run.smithi150.stdout:4955 write 0x673802b thru 0x674186a (0x9840 bytes) 2024-09-21T23:11:35.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:35.442 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:35.435+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:35.442 INFO:teuthology.orchestra.run.smithi150.stdout:4956 punch from 0xf8e891 to 0xf93b1a, (0x5289 bytes) 2024-09-21T23:11:35.442 INFO:teuthology.orchestra.run.smithi150.stdout:4957 read 0x33a5171 thru 0x33b1d51 (0xcbe1 bytes) 2024-09-21T23:11:35.463 INFO:teuthology.orchestra.run.smithi150.stdout:4958 writesame 0xb4b9986 thru 0xb4ba473 data_size 0x1(0xaee bytes) 2024-09-21T23:11:35.486 INFO:teuthology.orchestra.run.smithi150.stdout:4959 trunc from 0xb4ba474 to 0x627138 2024-09-21T23:11:35.503 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:35.522 INFO:teuthology.orchestra.run.smithi150.stdout:4962 trunc from 0x627138 to 0x8b27bb3 2024-09-21T23:11:35.534 INFO:teuthology.orchestra.run.smithi150.stdout:4963 punch from 0x7263bc to 0x735b80, (0xf7c4 bytes) 2024-09-21T23:11:35.541 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:35.537+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:35.547 INFO:teuthology.orchestra.run.smithi150.stdout:4965 write 0x8ca3b37 thru 0x8cacd6d (0x9237 bytes) 2024-09-21T23:11:35.551 INFO:teuthology.orchestra.run.smithi150.stdout:4968 clone 46 order 19 su 131072 sc 5 2024-09-21T23:11:35.571 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:35.571 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:35.572 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:35.572 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:35.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:35.642 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:35.638+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:35.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:11:35.743 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:35.738+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:35.756 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:35.756 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:35.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:35.845 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:35.841+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:35.852 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone45 from 0x8cacd6e (overlap 0x23b89e) to 0x11dc4f 2024-09-21T23:11:35.908 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:35.908 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:35.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:35.908 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:35.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.908 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:11:35.908 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 57.179454288999999, 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:11:35.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:11:35.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:11:35.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:35.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.914 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.532140796, 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:11:35.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.532108121, 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:11:35.919 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:35.923 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.532084786, 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:11:35.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:11:35.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.532052221000001, 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531964599, 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.935 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531845852, 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.939 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531821087000001, 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:11:35.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:35.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.943 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531799939999999, 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.944 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:11:35.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:11:35.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:11:35.947 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531742243, 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.948 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:11:35.951 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531707362000001, 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.952 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:35.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:11:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:11:35.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531681837000001, 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.956 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:11:35.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531610650999999, 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.960 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.964 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531574805, 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:35.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:11:35.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:11:35.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531557712, 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.972 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531541701, 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:11:35.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531484453000001, 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.977 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:35.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:11:35.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:11:35.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531403087999999, 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.981 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:11:35.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:11:35.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531371538, 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:35.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:11:35.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:35.989 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.531224641, 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:35.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:35.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:35.994 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:35.994 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:35.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:35.995 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:35.989+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:36.096 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:36.093+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:36.173 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:36.173 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:36.173 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:36.173 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:36.188 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:36.236 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:36.233+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:36.257 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-21T23:11:36.298 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:36.294+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:36.357 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:36.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:36.358 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:36.358 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:36.358 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:36.371 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:36.416 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:36.413+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:36.479 INFO:teuthology.orchestra.run.smithi150.stdout:4969 punch from 0x8b98d63 to 0x8b9f049, (0x62e6 bytes) 2024-09-21T23:11:36.490 INFO:teuthology.orchestra.run.smithi150.stdout:4970 writesame 0x434c65c thru 0x435a70a data_size 0x63(0xe0af bytes) 2024-09-21T23:11:36.498 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:36.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.311599665000003, 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:11:36.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:11:36.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.311496579000007, 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:11:36.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.311429230000002, 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.507 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:11:36.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.236242293999993, 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:11:36.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:11:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.236186525999997, 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:11:36.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:36.519 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.236154987000006, 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.523 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.236122237999993, 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.527 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.236095344999995, 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.528 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.236070072000004, 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:11:36.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:11:36.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.236038945000004, 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:11:36.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:36.540 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.235395589000007, 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:36.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230209661999993, 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.548 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230188178999995, 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:36.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:11:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:11:36.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.552 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230163892999997, 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:36.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:11:36.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:11:36.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230146226000002, 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:11:36.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:11:36.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230122640999994, 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:11:36.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230106625999994, 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.565 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:36.569 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230092795000004, 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:11:36.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:11:36.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.230061434000007, 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.574 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 89.229993011999994, 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:36.578 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:36.582 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:36.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:4974 writesame 0x55c53d1 thru 0x55d0eb4 data_size 0x36(0xbae4 bytes) 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:4975 read 0x3497e83 thru 0x3498e12 (0xf90 bytes) 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:4976 write 0x9db6fd7 thru 0x9dbe6d7 (0x7701 bytes) 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:4977 trunc from 0x9dbe6d8 to 0x3e9b14b 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:4979 punch from 0x3768d9b to 0x377213d, (0x93a2 bytes) 2024-09-21T23:11:36.585 INFO:teuthology.orchestra.run.smithi150.stdout:4980 write 0x47a0a0 thru 0x485dbc (0xbd1d bytes) 2024-09-21T23:11:36.586 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:36.578+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:36.602 INFO:teuthology.orchestra.run.smithi150.stdout:4981 trunc from 0x3e9b14b to 0xb461315 2024-09-21T23:11:36.639 INFO:teuthology.orchestra.run.smithi150.stdout:4982 trunc from 0xb461315 to 0x2663f91 2024-09-21T23:11:36.665 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:11:36.682 INFO:teuthology.orchestra.run.smithi150.stdout:4983 read 0x2003a74 thru 0x200c555 (0x8ae2 bytes) 2024-09-21T23:11:36.684 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:36.682+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:36.702 INFO:teuthology.orchestra.run.smithi150.stdout:4984 writesame 0x7f3899e thru 0x7f41a2f data_size 0x5(0x9092 bytes) 2024-09-21T23:11:36.702 INFO:teuthology.orchestra.run.smithi150.stdout:4986 read 0x11416c5 thru 0x114cf52 (0xb88e bytes) 2024-09-21T23:11:36.706 INFO:teuthology.orchestra.run.smithi150.stdout:4987 punch from 0x76f6ebd to 0x76febc6, (0x7d09 bytes) 2024-09-21T23:11:36.706 INFO:teuthology.orchestra.run.smithi150.stdout:4988 write 0x3fb0148 thru 0x3fb3edd (0x3d96 bytes) 2024-09-21T23:11:36.711 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:36.711 INFO:teuthology.orchestra.run.smithi150.stdout:4992 writesame 0x59d3719 thru 0x59da82e data_size 0x1e(0x7116 bytes) 2024-09-21T23:11:36.716 INFO:teuthology.orchestra.run.smithi150.stdout:4993 read 0x2e3a451 thru 0x2e44723 (0xa2d3 bytes) 2024-09-21T23:11:36.716 INFO:teuthology.orchestra.run.smithi150.stdout:4994 punch from 0x7069243 to 0x706b01a, (0x1dd7 bytes) 2024-09-21T23:11:36.717 INFO:teuthology.orchestra.run.smithi150.stdout:4995 read 0x784e1d6 thru 0x784f9aa (0x17d5 bytes) 2024-09-21T23:11:36.717 INFO:teuthology.orchestra.run.smithi150.stdout:4996 write 0x73bdfe thru 0x7486fd (0xc900 bytes) 2024-09-21T23:11:36.723 INFO:teuthology.orchestra.run.smithi150.stdout:4997 write 0x6deb130 thru 0x6debc72 (0xb43 bytes) 2024-09-21T23:11:36.727 INFO:teuthology.orchestra.run.smithi150.stdout:4998 writesame 0x3c70a41 thru 0x3c76345 data_size 0x1(0x5905 bytes) 2024-09-21T23:11:36.749 INFO:teuthology.orchestra.run.smithi150.stdout:5000 punch from 0x2c19477 to 0x2c22135, (0x8cbe bytes) 2024-09-21T23:11:36.762 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:36.762 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:36.762 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:36.771 INFO:teuthology.orchestra.run.smithi150.stdout:5001 writesame 0xc2f6ecd thru 0xc30014f data_size 0x1(0x9283 bytes) 2024-09-21T23:11:36.772 INFO:teuthology.orchestra.run.smithi150.stdout:5003 write 0x62775d5 thru 0x6280b0c (0x9538 bytes) 2024-09-21T23:11:36.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:36.779 INFO:teuthology.orchestra.run.smithi150.stdout:5004 write 0x4fe8ce3 thru 0x4fea6ff (0x1a1d bytes) 2024-09-21T23:11:36.785 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:36.782+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:36.786 INFO:teuthology.orchestra.run.smithi150.stdout:5005 write 0x9b9010e thru 0x9b95e0b (0x5cfe bytes) 2024-09-21T23:11:36.808 INFO:teuthology.orchestra.run.smithi150.stdout:5006 writesame 0xe01c015 thru 0xe028112 data_size 0x7(0xc0fe bytes) 2024-09-21T23:11:36.808 INFO:teuthology.orchestra.run.smithi150.stdout:5007 write 0xb476996 thru 0xb47a0e5 (0x3750 bytes) 2024-09-21T23:11:36.827 INFO:teuthology.orchestra.run.smithi150.stdout:5008 writesame 0xea305cd thru 0xea35cfa data_size 0x1(0x572e bytes) 2024-09-21T23:11:36.827 INFO:teuthology.orchestra.run.smithi150.stdout:5009 punch from 0x1f4b879 to 0x1f551a8, (0x992f bytes) 2024-09-21T23:11:36.827 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:36.835 INFO:teuthology.orchestra.run.smithi150.stdout:5013 write 0xea549be thru 0xea62af9 (0xe13c bytes) 2024-09-21T23:11:36.843 INFO:teuthology.orchestra.run.smithi150.stdout:5014 write 0xa466d03 thru 0xa469694 (0x2992 bytes) 2024-09-21T23:11:36.868 INFO:teuthology.orchestra.run.smithi150.stdout:5016 trunc from 0xea62afa to 0x7efd5b1 2024-09-21T23:11:36.878 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:36.886 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:36.881+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:36.906 INFO:teuthology.orchestra.run.smithi150.stdout:5018 writesame 0xa01b3a5 thru 0xa01bfdb data_size 0x1(0xc37 bytes) 2024-09-21T23:11:36.906 INFO:teuthology.orchestra.run.smithi150.stdout:5019 write 0x66d83d3 thru 0x66e22be (0x9eec bytes) 2024-09-21T23:11:36.908 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:36.908 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:36.908 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:36.908 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:36.908 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:36.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:36.930 INFO:teuthology.orchestra.run.smithi150.stdout:5020 trunc from 0xa01bfdc to 0xa8d224 2024-09-21T23:11:36.958 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:36.958 INFO:teuthology.orchestra.run.smithi150.stdout:5022 read 0x8e202f thru 0x8e6298 (0x426a bytes) 2024-09-21T23:11:36.979 INFO:teuthology.orchestra.run.smithi150.stdout:5024 writesame 0x9408650 thru 0x940d621 data_size 0x2(0x4fd2 bytes) 2024-09-21T23:11:36.989 INFO:teuthology.orchestra.run.smithi150.stdout:5025 trunc from 0x940d622 to 0x332aa2c 2024-09-21T23:11:36.996 INFO:teuthology.orchestra.run.smithi150.stdout:5027 read 0x1a11a96 thru 0x1a17f23 (0x648e bytes) 2024-09-21T23:11:36.996 INFO:teuthology.orchestra.run.smithi150.stdout:5028 punch from 0xebcce0 to 0xeca1ae, (0xd4ce bytes) 2024-09-21T23:11:36.996 INFO:teuthology.orchestra.run.smithi150.stdout:5031 read 0x21e459 thru 0x21edf7 (0x99f bytes) 2024-09-21T23:11:36.997 INFO:teuthology.orchestra.run.smithi150.stdout:5032 read 0x32b5917 thru 0x32c1996 (0xc080 bytes) 2024-09-21T23:11:37.008 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:37.005+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:37.014 INFO:teuthology.orchestra.run.smithi150.stdout:5034 write 0xa6a52de thru 0xa6b2179 (0xce9c bytes) 2024-09-21T23:11:37.018 INFO:teuthology.orchestra.run.smithi150.stdout:5035 read 0x8fdc056 thru 0x8fe137b (0x5326 bytes) 2024-09-21T23:11:37.018 INFO:teuthology.orchestra.run.smithi150.stdout:5037 write 0x16223bc thru 0x162f855 (0xd49a bytes) 2024-09-21T23:11:37.022 INFO:teuthology.orchestra.run.smithi150.stdout:5038 write 0x2d079d3 thru 0x2d151ba (0xd7e8 bytes) 2024-09-21T23:11:37.026 INFO:teuthology.orchestra.run.smithi150.stdout:5039 punch from 0x1d5a14d to 0x1d5be73, (0x1d26 bytes) 2024-09-21T23:11:37.040 INFO:teuthology.orchestra.run.smithi150.stdout:5041 writesame 0xeaec331 thru 0xeaf7062 data_size 0x7(0xad32 bytes) 2024-09-21T23:11:37.040 INFO:teuthology.orchestra.run.smithi150.stdout:5042 writesame 0x7a2fbee thru 0x7a31eb5 data_size 0x2a(0x22c8 bytes) 2024-09-21T23:11:37.042 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:37.042 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:37.044 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.044 INFO:teuthology.orchestra.run.smithi150.stdout:5044 punch from 0x6473e60 to 0x647b242, (0x73e2 bytes) 2024-09-21T23:11:37.044 INFO:teuthology.orchestra.run.smithi150.stdout:5045 write 0x733ea42 thru 0x734bc17 (0xd1d6 bytes) 2024-09-21T23:11:37.058 INFO:teuthology.orchestra.run.smithi150.stdout:5046 trunc from 0xeaf7063 to 0x86d0f5e 2024-09-21T23:11:37.060 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:37.060 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:37.071 INFO:teuthology.orchestra.run.smithi150.stdout:5048 punch from 0x223c96c to 0x2241bde, (0x5272 bytes) 2024-09-21T23:11:37.071 INFO:teuthology.orchestra.run.smithi150.stdout:5049 write 0x7b37d57 thru 0x7b3ce32 (0x50dc bytes) 2024-09-21T23:11:37.075 INFO:teuthology.orchestra.run.smithi150.stdout:5050 read 0x118051d thru 0x1186a3d (0x6521 bytes) 2024-09-21T23:11:37.075 INFO:teuthology.orchestra.run.smithi150.stdout:5051 write 0x75713f6 thru 0x757dc71 (0xc87c bytes) 2024-09-21T23:11:37.078 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:37.078 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:37.087 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:37.082+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:37.092 INFO:teuthology.orchestra.run.smithi150.stdout:5052 writesame 0xc588b4e thru 0xc58aa5e data_size 0xb(0x1f11 bytes) 2024-09-21T23:11:37.092 INFO:teuthology.orchestra.run.smithi150.stdout:5053 trunc from 0xc58aa5f to 0xc87e484 2024-09-21T23:11:37.095 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 35.702808570999998, 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71501 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7200768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232863505999999, 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069501725, 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71501 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860150+0000", 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-07 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860152+0000", 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9870000000000002e-06 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860158+0000", 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860162+0000", 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9180000000000002e-06 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929487+0000", 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069324330000000003 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929580+0000", 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2839999999999999e-05 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929651+0000", 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1416999999999997e-05 2024-09-21T23:11:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232843702, 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232820546999999, 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.23279833, 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:11:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:11:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:11:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232777719, 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:11:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232758094999999, 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:11:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232737158000001, 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:11:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232715195000001, 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:11:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232695770999999, 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.23267663, 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:11:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.23265018, 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:11:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:11:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.23263066, 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:11:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232607853999999, 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:11:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:11:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232587382, 2024-09-21T23:11:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232511617, 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.232487945000001, 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:11:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:11:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:11:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.227572774, 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:11:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.227548047999999, 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:11:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 15.227523473, 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:37.179 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:37.180 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:37.180 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:37.180 INFO:teuthology.orchestra.run.smithi150.stdout:5054 read 0x863433 thru 0x86e99c (0xb56a bytes) 2024-09-21T23:11:37.180 INFO:teuthology.orchestra.run.smithi150.stdout:5055 writesame 0x4fdbfd9 thru 0x4fe3669 data_size 0x1(0x7691 bytes) 2024-09-21T23:11:37.181 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:37.181 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:37.182 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5057 writesame 0x91f8ad thru 0x922dc4 data_size 0x4(0x3518 bytes) 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5060 write 0xa232291 thru 0xa2380f7 (0x5e67 bytes) 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5061 write 0x1c2d9bb thru 0x1c35994 (0x7fda bytes) 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5064 read 0xf269b2 thru 0xf2982d (0x2e7c bytes) 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5065 read 0x8e364ac thru 0x8e3dcbc (0x7811 bytes) 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5066 read 0x4d917f3 thru 0x4d93733 (0x1f41 bytes) 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5067 read 0xb3f7ff3 thru 0xb401948 (0x9956 bytes) 2024-09-21T23:11:37.183 INFO:teuthology.orchestra.run.smithi150.stdout:5068 read 0x765a80f thru 0x766917c (0xe96e bytes) 2024-09-21T23:11:37.184 INFO:teuthology.orchestra.run.smithi150.stdout:5070 read 0x9e1d434 thru 0x9e202f0 (0x2ebd bytes) 2024-09-21T23:11:37.184 INFO:teuthology.orchestra.run.smithi150.stdout:5071 punch from 0x5c30e96 to 0x5c40160, (0xf2ca bytes) 2024-09-21T23:11:37.184 INFO:teuthology.orchestra.run.smithi150.stdout:5074 writesame 0x6a4ea75 thru 0x6a5e3a3 data_size 0xd(0xf92f bytes) 2024-09-21T23:11:37.184 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:37.184 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:37.185 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.185 INFO:teuthology.orchestra.run.smithi150.stdout:5077 write 0x18c5cb5 thru 0x18d2372 (0xc6be bytes) 2024-09-21T23:11:37.185 INFO:teuthology.orchestra.run.smithi150.stdout:5079 read 0x5c1b517 thru 0x5c27218 (0xbd02 bytes) 2024-09-21T23:11:37.185 INFO:teuthology.orchestra.run.smithi150.stdout:5080 trunc from 0xc87e484 to 0x9678e40 2024-09-21T23:11:37.185 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:37.185 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:37.186 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:37.186 INFO:teuthology.orchestra.run.smithi150.stdout:5081 write 0xcce7393 thru 0xcce9a86 (0x26f4 bytes) 2024-09-21T23:11:37.186 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:37.187 INFO:teuthology.orchestra.run.smithi150.stdout:5083 write 0x2a9e744 thru 0x2aadb9d (0xf45a bytes) 2024-09-21T23:11:37.187 INFO:teuthology.orchestra.run.smithi150.stdout:5084 writesame 0xb7e13e1 thru 0xb7e896b data_size 0x1(0x758b bytes) 2024-09-21T23:11:37.204 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.204 INFO:teuthology.orchestra.run.smithi150.stdout:5086 write 0x687e0db thru 0x6884290 (0x61b6 bytes) 2024-09-21T23:11:37.210 INFO:teuthology.orchestra.run.smithi150.stdout:5090 read 0xa99f0a0 thru 0xa9a45ac (0x550d bytes) 2024-09-21T23:11:37.223 INFO:teuthology.orchestra.run.smithi150.stdout:5092 write 0xda68c3b thru 0xda6950e (0x8d4 bytes) 2024-09-21T23:11:37.224 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:37.221+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:37.228 INFO:teuthology.orchestra.run.smithi150.stdout:5095 writesame 0x61f79e9 thru 0x620543d data_size 0x3(0xda55 bytes) 2024-09-21T23:11:37.251 INFO:teuthology.orchestra.run.smithi150.stdout:5097 trunc from 0xda6950f to 0x714f20a 2024-09-21T23:11:37.277 INFO:teuthology.orchestra.run.smithi150.stdout:5098 read 0x1a4e65f thru 0x1a5c6a9 (0xe04b bytes) 2024-09-21T23:11:37.278 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.278 INFO:teuthology.orchestra.run.smithi150.stdout:5100 read 0x33fe8e9 thru 0x340a7ed (0xbf05 bytes) 2024-09-21T23:11:37.278 INFO:teuthology.orchestra.run.smithi150.stdout:5101 read 0x3991c0e thru 0x399a4e7 (0x88da bytes) 2024-09-21T23:11:37.278 INFO:teuthology.orchestra.run.smithi150.stdout:5102 read 0x6f755e1 thru 0x6f84a2e (0xf44e bytes) 2024-09-21T23:11:37.278 INFO:teuthology.orchestra.run.smithi150.stdout:5104 punch from 0x20b522e to 0x20c0289, (0xb05b bytes) 2024-09-21T23:11:37.278 INFO:teuthology.orchestra.run.smithi150.stdout:5105 writesame 0x5133b42 thru 0x5138f33 data_size 0x23(0x53f2 bytes) 2024-09-21T23:11:37.287 INFO:teuthology.orchestra.run.smithi150.stdout:5108 trunc from 0x714f20a to 0x52d1d19 2024-09-21T23:11:37.289 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:37.286+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:37.313 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:37.313 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:37.314 INFO:teuthology.orchestra.run.smithi150.stdout:5109 trunc from 0x52d1d19 to 0x22b7e96 2024-09-21T23:11:37.331 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:37.359 INFO:teuthology.orchestra.run.smithi150.stdout:5110 write 0x666df87 thru 0x666f022 (0x109c bytes) 2024-09-21T23:11:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:5112 punch from 0x1cde847 to 0x1ce0aff, (0x22b8 bytes) 2024-09-21T23:11:37.372 INFO:teuthology.orchestra.run.smithi150.stdout:5113 trunc from 0x666f023 to 0x2011175 2024-09-21T23:11:37.390 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:37.385+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:37.405 INFO:teuthology.orchestra.run.smithi150.stdout:5115 trunc from 0x2011175 to 0x997ca8d 2024-09-21T23:11:37.418 INFO:teuthology.orchestra.run.smithi150.stdout:5117 write 0x89f305d thru 0x89fd4d7 (0xa47b bytes) 2024-09-21T23:11:37.425 INFO:teuthology.orchestra.run.smithi150.stdout:5118 write 0x36ea2e9 thru 0x36f3f69 (0x9c81 bytes) 2024-09-21T23:11:37.430 INFO:teuthology.orchestra.run.smithi150.stdout:5120 write 0x4bdf367 thru 0x4be3bed (0x4887 bytes) 2024-09-21T23:11:37.452 INFO:teuthology.orchestra.run.smithi150.stdout:5121 write 0xaabcba1 thru 0xaac538c (0x87ec bytes) 2024-09-21T23:11:37.460 INFO:teuthology.orchestra.run.smithi150.stdout:5122 punch from 0x303717e to 0x30428ae, (0xb730 bytes) 2024-09-21T23:11:37.461 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.461 INFO:teuthology.orchestra.run.smithi150.stdout:5125 write 0x3f0c51 thru 0x3f2bf0 (0x1fa0 bytes) 2024-09-21T23:11:37.484 INFO:teuthology.orchestra.run.smithi150.stdout:5128 write 0xea2c14f thru 0xea2e13b (0x1fed bytes) 2024-09-21T23:11:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:37.491 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:37.486+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:37.492 INFO:teuthology.orchestra.run.smithi150.stdout:5129 read 0x491725a thru 0x4917eb3 (0xc5a bytes) 2024-09-21T23:11:37.492 INFO:teuthology.orchestra.run.smithi150.stdout:5130 writesame 0x771714b thru 0x771fa25 data_size 0x5b(0x88db bytes) 2024-09-21T23:11:37.502 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:37.502 INFO:teuthology.orchestra.run.smithi150.stdout:5131 trunc from 0xea2e13c to 0xdd57fde 2024-09-21T23:11:37.525 INFO:teuthology.orchestra.run.smithi150.stdout:5132 punch from 0x5a48076 to 0x5a55dbc, (0xdd46 bytes) 2024-09-21T23:11:37.525 INFO:teuthology.orchestra.run.smithi150.stdout:5134 read 0xae23914 thru 0xae29b67 (0x6254 bytes) 2024-09-21T23:11:37.525 INFO:teuthology.orchestra.run.smithi150.stdout:5136 writesame 0x9f75591 thru 0x9f79010 data_size 0x68(0x3a80 bytes) 2024-09-21T23:11:37.532 INFO:teuthology.orchestra.run.smithi150.stdout:5137 punch from 0x95582f5 to 0x9559618, (0x1323 bytes) 2024-09-21T23:11:37.532 INFO:teuthology.orchestra.run.smithi150.stdout:5139 punch from 0x3f50a1b to 0x3f5a538, (0x9b1d bytes) 2024-09-21T23:11:37.532 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.533 INFO:teuthology.orchestra.run.smithi150.stdout:5141 trunc from 0xdd57fde to 0xd16a370 2024-09-21T23:11:37.545 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:37.546 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:37.546 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:11:37.546 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:11:37.546 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:11:37.546 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:11:37.547 INFO:teuthology.orchestra.run.smithi150.stdout:5142 read 0x397d696 thru 0x397e044 (0x9af bytes) 2024-09-21T23:11:37.547 INFO:teuthology.orchestra.run.smithi150.stdout:5143 write 0x9847691 thru 0x9855529 (0xde99 bytes) 2024-09-21T23:11:37.567 INFO:teuthology.orchestra.run.smithi150.stdout:5144 trunc from 0xd16a370 to 0x4283898 2024-09-21T23:11:37.590 INFO:teuthology.orchestra.run.smithi150.stdout:5145 read 0x2bd121d thru 0x2be0b4f (0xf933 bytes) 2024-09-21T23:11:37.590 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.591 INFO:teuthology.orchestra.run.smithi150.stdout:5147 read 0xa81f48 thru 0xa89b06 (0x7bbf bytes) 2024-09-21T23:11:37.591 INFO:teuthology.orchestra.run.smithi150.stdout:5148 punch from 0x3be49a0 to 0x3bf4343, (0xf9a3 bytes) 2024-09-21T23:11:37.592 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:37.586+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:37.597 INFO:teuthology.orchestra.run.smithi150.stdout:5149 trunc from 0x4283898 to 0x866534c 2024-09-21T23:11:37.608 INFO:teuthology.orchestra.run.smithi150.stdout:5150 read 0x3af356e thru 0x3af37d2 (0x265 bytes) 2024-09-21T23:11:37.608 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:37.609 INFO:teuthology.orchestra.run.smithi150.stdout:5155 punch from 0x49189ea to 0x492771b, (0xed31 bytes) 2024-09-21T23:11:37.609 INFO:teuthology.orchestra.run.smithi150.stdout:5156 read 0x5900b9c thru 0x59079b8 (0x6e1d bytes) 2024-09-21T23:11:37.630 INFO:teuthology.orchestra.run.smithi150.stdout:5157 writesame 0xdbc3790 thru 0xdbca0a0 data_size 0x1(0x6911 bytes) 2024-09-21T23:11:37.630 INFO:teuthology.orchestra.run.smithi150.stdout:5158 read 0xb63fc3b thru 0xb649c71 (0xa037 bytes) 2024-09-21T23:11:37.630 INFO:teuthology.orchestra.run.smithi150.stdout:5159 punch from 0xd8daa2f to 0xd8e489d, (0x9e6e bytes) 2024-09-21T23:11:37.630 INFO:teuthology.orchestra.run.smithi150.stdout:5160 writesame 0x6d250dd thru 0x6d31b40 data_size 0x2(0xca64 bytes) 2024-09-21T23:11:37.654 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:37.654 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:37.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:37.673 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "" 2024-09-21T23:11:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:37.687 INFO:teuthology.orchestra.run.smithi150.stdout:5163 trunc from 0xdbca0a1 to 0x10d5924 2024-09-21T23:11:37.693 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:37.689+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:37.731 INFO:teuthology.orchestra.run.smithi150.stdout:5166 write 0x7a37248 thru 0x7a408de (0x9697 bytes) 2024-09-21T23:11:37.750 INFO:teuthology.orchestra.run.smithi150.stdout:5167 write 0x905df65 thru 0x906aa2e (0xcaca bytes) 2024-09-21T23:11:37.759 INFO:teuthology.orchestra.run.smithi150.stdout:5168 read 0x4fb3218 thru 0x4fb3651 (0x43a bytes) 2024-09-21T23:11:37.759 INFO:teuthology.orchestra.run.smithi150.stdout:5170 write 0x50ee3d4 thru 0x50f4df3 (0x6a20 bytes) 2024-09-21T23:11:37.772 INFO:teuthology.orchestra.run.smithi150.stdout:5172 trunc from 0x906aa2f to 0x4a24f42 2024-09-21T23:11:37.794 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:37.789+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:37.802 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:37.802 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:37.811 INFO:teuthology.orchestra.run.smithi150.stdout:5173 write 0x6b13adc thru 0x6b1f3e8 (0xb90d bytes) 2024-09-21T23:11:37.817 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:37.844 INFO:teuthology.orchestra.run.smithi150.stdout:5175 write 0xd78033b thru 0xd782389 (0x204f bytes) 2024-09-21T23:11:37.852 INFO:teuthology.orchestra.run.smithi150.stdout:5177 read 0x634cf77 thru 0x634e1fe (0x1288 bytes) 2024-09-21T23:11:37.852 INFO:teuthology.orchestra.run.smithi150.stdout:5178 read 0x5468121 thru 0x546cdbc (0x4c9c bytes) 2024-09-21T23:11:37.852 INFO:teuthology.orchestra.run.smithi150.stdout:5179 write 0x8900def thru 0x890f4be (0xe6d0 bytes) 2024-09-21T23:11:37.858 INFO:teuthology.orchestra.run.smithi150.stdout:5180 writesame 0x410b45a thru 0x41155b3 data_size 0x13(0xa15a bytes) 2024-09-21T23:11:37.868 INFO:teuthology.orchestra.run.smithi150.stdout:5181 writesame 0x5378f0d thru 0x537aef0 data_size 0x1a(0x1fe4 bytes) 2024-09-21T23:11:37.874 INFO:teuthology.orchestra.run.smithi150.stdout:5183 read 0x41ef92b thru 0x41f9e3c (0xa512 bytes) 2024-09-21T23:11:37.895 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:37.890+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:37.899 INFO:teuthology.orchestra.run.smithi150.stdout:5184 trunc from 0xd78238a to 0x1cbe955 2024-09-21T23:11:37.922 INFO:teuthology.orchestra.run.smithi150.stdout:5185 punch from 0xd7ff7b to 0xd8591a, (0x599f bytes) 2024-09-21T23:11:37.940 INFO:teuthology.orchestra.run.smithi150.stdout:5187 write 0x3f29bda thru 0x3f3102f (0x7456 bytes) 2024-09-21T23:11:37.947 INFO:teuthology.orchestra.run.smithi150.stdout:5188 read 0x3774593 thru 0x378279b (0xe209 bytes) 2024-09-21T23:11:37.947 INFO:teuthology.orchestra.run.smithi150.stdout:5189 read 0x2485ea9 thru 0x248e405 (0x855d bytes) 2024-09-21T23:11:37.979 INFO:teuthology.orchestra.run.smithi150.stdout:5190 write 0xe4ff6c1 thru 0xe508b08 (0x9448 bytes) 2024-09-21T23:11:37.980 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:37.980 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:37.989 INFO:teuthology.orchestra.run.smithi150.stdout:5192 read 0x8e2109b thru 0x8e2e6f8 (0xd65e bytes) 2024-09-21T23:11:37.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:37.997 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:37.993+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:38.004 INFO:teuthology.orchestra.run.smithi150.stdout:5193 trunc from 0xe508b09 to 0x7377bc3 2024-09-21T23:11:38.025 INFO:teuthology.orchestra.run.smithi150.stdout:5194 read 0x26f7e4 thru 0x2722e5 (0x2b02 bytes) 2024-09-21T23:11:38.026 INFO:teuthology.orchestra.run.smithi150.stdout:5196 writesame 0x67813a8 thru 0x678222b data_size 0x2(0xe84 bytes) 2024-09-21T23:11:38.044 INFO:teuthology.orchestra.run.smithi150.stdout:5197 trunc from 0x7377bc3 to 0x12b362c 2024-09-21T23:11:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:5198 write 0x2257772 thru 0x225d361 (0x5bf0 bytes) 2024-09-21T23:11:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:5199 punch from 0x1df4362 to 0x1dfbf2d, (0x7bcb bytes) 2024-09-21T23:11:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:5200 trunc from 0x225d362 to 0x3f8bd1a 2024-09-21T23:11:38.097 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:38.093+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:5202 write 0x1d92d3a thru 0x1d9504e (0x2315 bytes) 2024-09-21T23:11:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:5203 trunc from 0x3f8bd1a to 0x776cb71 2024-09-21T23:11:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:5204 trunc from 0x776cb71 to 0x6b658d3 2024-09-21T23:11:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:5205 read 0x573e856 thru 0x574a80d (0xbfb8 bytes) 2024-09-21T23:11:38.143 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:5206 write 0x9e52e7f thru 0x9e5dead (0xb02f bytes) 2024-09-21T23:11:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:5207 read 0x1840da5 thru 0x18424f5 (0x1751 bytes) 2024-09-21T23:11:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:5209 read 0x26ce5c0 thru 0x26d21c0 (0x3c01 bytes) 2024-09-21T23:11:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:5212 trunc from 0x9e5deae to 0x2b749b7 2024-09-21T23:11:38.198 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:38.193+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:38.238 INFO:teuthology.orchestra.run.smithi150.stdout:5213 write 0xb92c968 thru 0xb935d76 (0x940f bytes) 2024-09-21T23:11:38.246 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:38.246 INFO:teuthology.orchestra.run.smithi150.stdout:5215 writesame 0x370bf1a thru 0x3711c81 data_size 0x3d(0x5d68 bytes) 2024-09-21T23:11:38.253 INFO:teuthology.orchestra.run.smithi150.stdout:5217 write 0x3f35be thru 0x3f8c2c (0x566f bytes) 2024-09-21T23:11:38.275 INFO:teuthology.orchestra.run.smithi150.stdout:5218 writesame 0xe553d4e thru 0xe5630dd data_size 0x48(0xf390 bytes) 2024-09-21T23:11:38.275 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:38.275 INFO:teuthology.orchestra.run.smithi150.stdout:5220 punch from 0x6de2cab to 0x6df1e7a, (0xf1cf bytes) 2024-09-21T23:11:38.275 INFO:teuthology.orchestra.run.smithi150.stdout:5221 read 0x5f08637 thru 0x5f0d124 (0x4aee bytes) 2024-09-21T23:11:38.282 INFO:teuthology.orchestra.run.smithi150.stdout:5222 trunc from 0xe5630de to 0xa727532 2024-09-21T23:11:38.293 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:38.293 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:38.298 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:38.294+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:5225 read 0x3c919ca thru 0x3c91d26 (0x35d bytes) 2024-09-21T23:11:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:5226 write 0x6c18847 thru 0x6c1fbe8 (0x73a2 bytes) 2024-09-21T23:11:38.309 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:38.312 INFO:teuthology.orchestra.run.smithi150.stdout:5228 read 0x310b6c5 thru 0x3110328 (0x4c64 bytes) 2024-09-21T23:11:38.312 INFO:teuthology.orchestra.run.smithi150.stdout:5229 write 0x93af93c thru 0x93bd642 (0xdd07 bytes) 2024-09-21T23:11:38.319 INFO:teuthology.orchestra.run.smithi150.stdout:5231 read 0x8706e6b thru 0x870f71c (0x88b2 bytes) 2024-09-21T23:11:38.337 INFO:teuthology.orchestra.run.smithi150.stdout:5233 write 0xc7975e1 thru 0xc7a14ff (0x9f1f bytes) 2024-09-21T23:11:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:5234 punch from 0x5392d71 to 0x53a1091, (0xe320 bytes) 2024-09-21T23:11:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:5237 punch from 0x88d6227 to 0x88db56f, (0x5348 bytes) 2024-09-21T23:11:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:5238 punch from 0xbc3fd08 to 0xbc41e2a, (0x2122 bytes) 2024-09-21T23:11:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:5241 read 0x39ef45d thru 0x39f16db (0x227f bytes) 2024-09-21T23:11:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:5242 read 0x666d2e2 thru 0x6674325 (0x7044 bytes) 2024-09-21T23:11:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:5244 read 0x6c25c7 thru 0x6c7aca (0x5504 bytes) 2024-09-21T23:11:38.367 INFO:teuthology.orchestra.run.smithi150.stdout:5245 trunc from 0xc7a1500 to 0xf92f41 2024-09-21T23:11:38.388 INFO:teuthology.orchestra.run.smithi150.stdout:5246 read 0xcf7d36 thru 0xcf8fd7 (0x12a2 bytes) 2024-09-21T23:11:38.393 INFO:teuthology.orchestra.run.smithi150.stdout:5247 trunc from 0xf92f41 to 0x3ad5eab 2024-09-21T23:11:38.399 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:38.393+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:38.404 INFO:teuthology.orchestra.run.smithi150.stdout:5248 read 0x139370b thru 0x139de2f (0xa725 bytes) 2024-09-21T23:11:38.404 INFO:teuthology.orchestra.run.smithi150.stdout:5249 writesame 0x1c726db thru 0x1c74530 data_size 0xb(0x1e56 bytes) 2024-09-21T23:11:38.415 INFO:teuthology.orchestra.run.smithi150.stdout:5252 trunc from 0x3ad5eab to 0xb03e27 2024-09-21T23:11:38.442 INFO:teuthology.orchestra.run.smithi150.stdout:5253 trunc from 0xb03e27 to 0x24de160 2024-09-21T23:11:38.456 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:38.457 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:38.475 INFO:teuthology.orchestra.run.smithi150.stdout:5255 write 0x68827c2 thru 0x6888b78 (0x63b7 bytes) 2024-09-21T23:11:38.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:38.483 INFO:teuthology.orchestra.run.smithi150.stdout:5256 read 0x1880402 thru 0x18859ec (0x55eb bytes) 2024-09-21T23:11:38.483 INFO:teuthology.orchestra.run.smithi150.stdout:5257 read 0x3e0e778 thru 0x3e124d0 (0x3d59 bytes) 2024-09-21T23:11:38.492 INFO:teuthology.orchestra.run.smithi150.stdout:5258 trunc from 0x6888b79 to 0x5dff16 2024-09-21T23:11:38.499 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:38.494+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:38.526 INFO:teuthology.orchestra.run.smithi150.stdout:5260 punch from 0x4afa4c to 0x4bf718, (0xfccc bytes) 2024-09-21T23:11:38.541 INFO:teuthology.orchestra.run.smithi150.stdout:5261 writesame 0x237c248 thru 0x237d2d0 data_size 0x11(0x1089 bytes) 2024-09-21T23:11:38.562 INFO:teuthology.orchestra.run.smithi150.stdout:5262 write 0x84d244d thru 0x84d3b24 (0x16d8 bytes) 2024-09-21T23:11:38.589 INFO:teuthology.orchestra.run.smithi150.stdout:5263 write 0xd3998cd thru 0xd39b003 (0x1737 bytes) 2024-09-21T23:11:38.595 INFO:teuthology.orchestra.run.smithi150.stdout:5265 clone 47 order 20 su 65536 sc 15 2024-09-21T23:11:38.601 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:38.597+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:38.615 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:38.615 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:38.629 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:38.689 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:11:38.721 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:38.718+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:38.801 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:38.801 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:38.803 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:38.797+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:38.817 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:38.904 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:38.898+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:38.925 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone46 from 0xd39b004 (overlap 0x5dff16) to 0x3865da 2024-09-21T23:11:38.968 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:38.968 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:38.987 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:39.029 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:39.026+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:39.107 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:39.101+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:39.146 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:39.146 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:39.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:39.194 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:39.208 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:39.205+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:39.296 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:39.296 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:39.308 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:39.305+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:39.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:39.353 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:39.353 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:39.358 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-21T23:11:39.366 INFO:teuthology.orchestra.run.smithi150.stdout:5266 punch from 0x2d92554 to 0x2d9da6d, (0xb519 bytes) 2024-09-21T23:11:39.371 INFO:teuthology.orchestra.run.smithi150.stdout:5267 write 0xbc1805a thru 0xbc201af (0x8156 bytes) 2024-09-21T23:11:39.372 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:39.373 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:39.377 INFO:teuthology.orchestra.run.smithi150.stdout:5268 punch from 0xe6b291 to 0xe78c8b, (0xd9fa bytes) 2024-09-21T23:11:39.377 INFO:teuthology.orchestra.run.smithi150.stdout:5271 read 0x7338beb thru 0x733a852 (0x1c68 bytes) 2024-09-21T23:11:39.385 INFO:teuthology.orchestra.run.smithi150.stdout:5272 write 0xe2733e6 thru 0xe27728e (0x3ea9 bytes) 2024-09-21T23:11:39.395 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:39.395 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:39.407 INFO:teuthology.orchestra.run.smithi150.stdout:5273 trunc from 0xe27728f to 0x35dc83a 2024-09-21T23:11:39.408 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:39.402+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:39.421 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:39.437 INFO:teuthology.orchestra.run.smithi150.stdout:5274 read 0x18d1b87 thru 0x18dc43c (0xa8b6 bytes) 2024-09-21T23:11:39.443 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:39.443 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:39.451 INFO:teuthology.orchestra.run.smithi150.stdout:5275 write 0x88f85bc thru 0x88fbd39 (0x377e bytes) 2024-09-21T23:11:39.463 INFO:teuthology.orchestra.run.smithi150.stdout:5276 writesame 0x99ab385 thru 0x99aeeff data_size 0x1(0x3b7b bytes) 2024-09-21T23:11:39.463 INFO:teuthology.orchestra.run.smithi150.stdout:5277 read 0x1b649f5 thru 0x1b6eea3 (0xa4af bytes) 2024-09-21T23:11:39.463 INFO:teuthology.orchestra.run.smithi150.stdout:5278 read 0x65d894c thru 0x65e1d36 (0x93eb bytes) 2024-09-21T23:11:39.463 INFO:teuthology.orchestra.run.smithi150.stdout:5279 read 0x4a4027a thru 0x4a4aea5 (0xac2c bytes) 2024-09-21T23:11:39.464 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:39.464 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:39.464 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:39.470 INFO:teuthology.orchestra.run.smithi150.stdout:5281 write 0xa83b051 thru 0xa83bc34 (0xbe4 bytes) 2024-09-21T23:11:39.474 INFO:teuthology.orchestra.run.smithi150.stdout:5282 read 0x2b3ba0b thru 0x2b47a23 (0xc019 bytes) 2024-09-21T23:11:39.474 INFO:teuthology.orchestra.run.smithi150.stdout:5283 read 0x2abad6 thru 0x2adf57 (0x2482 bytes) 2024-09-21T23:11:39.475 INFO:teuthology.orchestra.run.smithi150.stdout:5284 writesame 0x1174b54 thru 0x117ebed data_size 0x3d(0xa09a bytes) 2024-09-21T23:11:39.479 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:39.479 INFO:teuthology.orchestra.run.smithi150.stdout:5287 punch from 0x4ceead3 to 0x4cf306c, (0x4599 bytes) 2024-09-21T23:11:39.480 INFO:teuthology.orchestra.run.smithi150.stdout:5288 trunc from 0xa83bc35 to 0x9e03d32 2024-09-21T23:11:39.483 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:39.483 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:39.486 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:39.486 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:39.486 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:39.486 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:39.486 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:39.497 INFO:teuthology.orchestra.run.smithi150.stdout:5289 trunc from 0x9e03d32 to 0xe479330 2024-09-21T23:11:39.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:39.502 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:39.502 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:39.504 INFO:teuthology.orchestra.run.smithi150.stdout:5291 punch from 0xae68750 to 0xae743d3, (0xbc83 bytes) 2024-09-21T23:11:39.504 INFO:teuthology.orchestra.run.smithi150.stdout:5292 write 0x119f4e thru 0x11dc2a (0x3cdd bytes) 2024-09-21T23:11:39.509 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:39.506+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:39.510 INFO:teuthology.orchestra.run.smithi150.stdout:5293 write 0x13d72ae thru 0x13dbae0 (0x4833 bytes) 2024-09-21T23:11:39.537 INFO:teuthology.orchestra.run.smithi150.stdout:5294 trunc from 0xe479330 to 0x2461a8 2024-09-21T23:11:39.599 INFO:teuthology.orchestra.run.smithi150.stdout:5295 writesame 0x7754c22 thru 0x775cb5d data_size 0x22(0x7f3c bytes) 2024-09-21T23:11:39.599 INFO:teuthology.orchestra.run.smithi150.stdout:5296 punch from 0x49d0973 to 0x49de1a9, (0xd836 bytes) 2024-09-21T23:11:39.599 INFO:teuthology.orchestra.run.smithi150.stdout:5298 writesame 0x207e6c1 thru 0x20878f4 data_size 0x6(0x9234 bytes) 2024-09-21T23:11:39.607 INFO:teuthology.orchestra.run.smithi150.stdout:5299 punch from 0x6be119d to 0x6bf0394, (0xf1f7 bytes) 2024-09-21T23:11:39.607 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:39.607 INFO:teuthology.orchestra.run.smithi150.stdout:5306 read 0x1bd6b7 thru 0x1c2975 (0x52bf bytes) 2024-09-21T23:11:39.608 INFO:teuthology.orchestra.run.smithi150.stdout:5307 read 0x479066c thru 0x479df68 (0xd8fd bytes) 2024-09-21T23:11:39.608 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:39.608 INFO:teuthology.orchestra.run.smithi150.stdout:5311 read 0x3906f4 thru 0x3996d1 (0x8fde bytes) 2024-09-21T23:11:39.609 INFO:teuthology.orchestra.run.smithi150.stdout:5312 clone 48 order 19 su 32768 sc 6 2024-09-21T23:11:39.609 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:39.605+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:39.629 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:39.629 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804934542000002, 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:11:39.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804863533999999, 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:11:39.634 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:11:39.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804799243000001, 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.639 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:11:39.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804768744, 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.643 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:11:39.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:11:39.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804620757999999, 2024-09-21T23:11:39.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804564252999999, 2024-09-21T23:11:39.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804509677999999, 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804454459999999, 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.660 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:11:39.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804399831000001, 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804280681000002, 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.668 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.672 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804203247, 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:11:39.676 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804137624999999, 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:11:39.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:11:39.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:11:39.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804073296999999, 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:11:39.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:11:39.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.804012209, 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:11:39.685 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.803951743999999, 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:11:39.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:11:39.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:11:39.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.693 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.803887545999999, 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:11:39.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:11:39.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:11:39.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.803813538, 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.698 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:39.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:11:39.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:11:39.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.803776112000001, 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.702 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:39.706 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.80372959, 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:39.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:11:39.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:11:39.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.803670095000001, 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.711 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:39.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:39.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:39.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:39.715 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:39.717 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:11:39.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:39.717 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:39.714+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:39.817 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:39.814+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:39.889 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:39.889 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:39.890 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:39.890 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:39.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:39.948 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:39.945+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:39.996 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone47 from 0x775cb5e (overlap 0x2461a8) to 0x3a35d 2024-09-21T23:11:40.018 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:40.014+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:40.038 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:40.038 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:40.038 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:40.038 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:40.038 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:40.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:40.120 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:40.114+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:40.220 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:40.217+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:40.223 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.492964841000003, 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:11:40.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:11:40.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845651348000001, 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:11:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:11:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845618673000001, 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:11:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:11:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845595337999999, 2024-09-21T23:11:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:11:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:11:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845562773000001, 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845475150999999, 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:11:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845356404, 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:11:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:11:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:11:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845331639000001, 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845310491999999, 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:11:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:11:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845252795, 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845217913999999, 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:11:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:11:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845192389000001, 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:11:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:11:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845121203000001, 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:11:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845085356999999, 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:11:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845068263999998, 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.845052252999999, 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:11:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:11:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.844995005000001, 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.844913640000001, 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.844882089999999, 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:11:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.844735192999998, 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:11:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.310 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:40.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:40.321 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:40.318+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:40.422 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:40.417+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:40.486 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:40.486 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:40.486 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:40.487 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:40.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:40.544 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:40.541+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:40.608 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-21T23:11:40.624 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:40.622+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:40.636 INFO:teuthology.orchestra.run.smithi150.stdout:5313 read 0x6e9e916 thru 0x6e9fdc2 (0x14ad bytes) 2024-09-21T23:11:40.637 INFO:teuthology.orchestra.run.smithi150.stdout:5314 read 0x4a36ca thru 0x4b1a9c (0xe3d3 bytes) 2024-09-21T23:11:40.639 INFO:teuthology.orchestra.run.smithi150.stdout:5315 read 0x607749f thru 0x6082159 (0xacbb bytes) 2024-09-21T23:11:40.670 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:40.670 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:40.670 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:40.670 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:40.670 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:40.684 INFO:teuthology.orchestra.run.smithi150.stdout:5316 writesame 0xee1294f thru 0xee19857 data_size 0x19(0x6f09 bytes) 2024-09-21T23:11:40.684 INFO:teuthology.orchestra.run.smithi150.stdout:5317 writesame 0x9de1444 thru 0x9debf7b data_size 0x4(0xab38 bytes) 2024-09-21T23:11:40.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:40.732 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:40.729+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:40.733 INFO:teuthology.orchestra.run.smithi150.stdout:5318 trunc from 0xee19858 to 0x2599bbd 2024-09-21T23:11:40.771 INFO:teuthology.orchestra.run.smithi150.stdout:5319 punch from 0xb4f496 to 0xb5a045, (0xabaf bytes) 2024-09-21T23:11:40.796 INFO:teuthology.orchestra.run.smithi150.stdout:5320 write 0x6d656ef thru 0x6d6f464 (0x9d76 bytes) 2024-09-21T23:11:40.805 INFO:teuthology.orchestra.run.smithi150.stdout:5321 punch from 0x4b50434 to 0x4b5ba75, (0xb641 bytes) 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.629120646999993, 2024-09-21T23:11:40.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:11:40.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.818 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.629017560999998, 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.819 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:11:40.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.628950212000007, 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.823 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.553763275999998, 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.827 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:40.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.553707508000002, 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.831 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:40.835 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.553675968999997, 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.839 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.553643219999998, 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:11:40.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.553616327, 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.844 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:11:40.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.553591053999995, 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:11:40.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.553559926999995, 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.853 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.857 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.552916570999997, 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.858 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:40.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:11:40.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.862 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547730643999998, 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.863 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547709161, 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.867 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:11:40.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:40.871 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547684875000002, 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:40.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:11:40.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547667208000007, 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:11:40.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:11:40.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:11:40.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:11:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547643622999999, 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:11:40.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.884 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547627607999999, 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:11:40.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:11:40.888 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547613776999995, 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.889 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547582415999997, 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:11:40.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.547513993999999, 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:40.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:11:40.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:11:40.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:11:40.902 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.903 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.903 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.903 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:40.903 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:40.903 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:40.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:40.905 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:40.901+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5322 trunc from 0x6d6f465 to 0xdbac2fb 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5323 write 0xe0a32b5 thru 0xe0ac064 (0x8db0 bytes) 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5324 trunc from 0xe0ac065 to 0xd1f4d33 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5326 read 0x582b47e thru 0x5838536 (0xd0b9 bytes) 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5327 read 0xad702ea thru 0xad76295 (0x5fac bytes) 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5328 read 0x4f1401c thru 0x4f1aa63 (0x6a48 bytes) 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5329 write 0x2809b09 thru 0x280c676 (0x2b6e bytes) 2024-09-21T23:11:40.906 INFO:teuthology.orchestra.run.smithi150.stdout:5333 punch from 0xb1a04a5 to 0xb1a2d0e, (0x2869 bytes) 2024-09-21T23:11:40.907 INFO:teuthology.orchestra.run.smithi150.stdout:5334 write 0x1546a70 thru 0x1549364 (0x28f5 bytes) 2024-09-21T23:11:40.923 INFO:teuthology.orchestra.run.smithi150.stdout:5335 trunc from 0xd1f4d33 to 0x504db1c 2024-09-21T23:11:40.949 INFO:teuthology.orchestra.run.smithi150.stdout:5336 write 0x1ea6355 thru 0x1eb1a4b (0xb6f7 bytes) 2024-09-21T23:11:40.964 INFO:teuthology.orchestra.run.smithi150.stdout:5337 read 0x458cc80 thru 0x458d606 (0x987 bytes) 2024-09-21T23:11:40.965 INFO:teuthology.orchestra.run.smithi150.stdout:5338 read 0x4dddb9e thru 0x4de59cd (0x7e30 bytes) 2024-09-21T23:11:40.965 INFO:teuthology.orchestra.run.smithi150.stdout:5339 read 0x4f43d1e thru 0x4f480ec (0x43cf bytes) 2024-09-21T23:11:40.965 INFO:teuthology.orchestra.run.smithi150.stdout:5341 read 0x482dfe8 thru 0x48397ce (0xb7e7 bytes) 2024-09-21T23:11:40.965 INFO:teuthology.orchestra.run.smithi150.stdout:5342 read 0x2c0ff7d thru 0x2c19350 (0x93d4 bytes) 2024-09-21T23:11:40.967 INFO:teuthology.orchestra.run.smithi150.stdout:5343 trunc from 0x504db1c to 0x5eaea49 2024-09-21T23:11:41.005 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:41.001+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:41.014 INFO:teuthology.orchestra.run.smithi150.stdout:5344 writesame 0xdf4193d thru 0xdf518c7 data_size 0x1(0xff8b bytes) 2024-09-21T23:11:41.014 INFO:teuthology.orchestra.run.smithi150.stdout:5345 write 0x5c9f85e thru 0x5ca4353 (0x4af6 bytes) 2024-09-21T23:11:41.022 INFO:teuthology.orchestra.run.smithi150.stdout:5347 punch from 0xdae904f to 0xdaf5cf4, (0xcca5 bytes) 2024-09-21T23:11:41.022 INFO:teuthology.orchestra.run.smithi150.stdout:5348 read 0x9a29769 thru 0x9a30110 (0x69a8 bytes) 2024-09-21T23:11:41.022 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:41.039 INFO:teuthology.orchestra.run.smithi150.stdout:5354 trunc from 0xdf518c8 to 0x74104c2 2024-09-21T23:11:41.068 INFO:teuthology.orchestra.run.smithi150.stdout:5356 write 0x88c499e thru 0x88d2a53 (0xe0b6 bytes) 2024-09-21T23:11:41.082 INFO:teuthology.orchestra.run.smithi150.stdout:5357 trunc from 0x88d2a54 to 0xa8613ba 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:92561 3.1a 3:5e91bc3d:::rbd_object_map.114926367aa2:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:41.083325+0000", 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.002937687, 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029827650000000001, 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.086 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 92561 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083325+0000", 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083325+0000", 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083326+0000", 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.367e-06 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083332+0000", 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1360000000000002e-06 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083349+0000", 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6796e-05 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083358+0000", 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6810000000000004e-06 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083488+0000", 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000130557 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.083745+0000", 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025672100000000001 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1", 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.084468+0000", 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000722607 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:41.086035+0000", 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015669690000000001 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.090 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.091 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:11:41.091 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:11:41.091 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:41.096 INFO:teuthology.orchestra.run.smithi150.stdout:5358 writesame 0x97fe7df thru 0x980dac4 data_size 0x1(0xf2e6 bytes) 2024-09-21T23:11:41.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:41.144 INFO:teuthology.orchestra.run.smithi150.stdout:5359 read 0x181091e thru 0x18180eb (0x77ce bytes) 2024-09-21T23:11:41.144 INFO:teuthology.orchestra.run.smithi150.stdout:5360 read 0xe1c0e3 thru 0xe20497 (0x43b5 bytes) 2024-09-21T23:11:41.144 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:41.141+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:41.145 INFO:teuthology.orchestra.run.smithi150.stdout:5361 punch from 0x765e5b0 to 0x766aa74, (0xc4c4 bytes) 2024-09-21T23:11:41.145 INFO:teuthology.orchestra.run.smithi150.stdout:5362 write 0x6cd3347 thru 0x6cdaceb (0x79a5 bytes) 2024-09-21T23:11:41.153 INFO:teuthology.orchestra.run.smithi150.stdout:5363 punch from 0x8aacd54 to 0x8ab066f, (0x391b bytes) 2024-09-21T23:11:41.158 INFO:teuthology.orchestra.run.smithi150.stdout:5364 trunc from 0xa8613ba to 0x9052898 2024-09-21T23:11:41.203 INFO:teuthology.orchestra.run.smithi150.stdout:5365 write 0xcf3793e thru 0xcf3d4b7 (0x5b7a bytes) 2024-09-21T23:11:41.206 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:41.201+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:41.214 INFO:teuthology.orchestra.run.smithi150.stdout:5367 trunc from 0xcf3d4b8 to 0xbeb2918 2024-09-21T23:11:41.257 INFO:teuthology.orchestra.run.smithi150.stdout:5368 trunc from 0xbeb2918 to 0x46c7388 2024-09-21T23:11:41.273 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:41.273 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:41.274 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:41.274 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:41.283 INFO:teuthology.orchestra.run.smithi150.stdout:5371 read 0x15a0d75 thru 0x15aff17 (0xf1a3 bytes) 2024-09-21T23:11:41.283 INFO:teuthology.orchestra.run.smithi150.stdout:5372 punch from 0x31a31a2 to 0x31af825, (0xc683 bytes) 2024-09-21T23:11:41.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:41.303 INFO:teuthology.orchestra.run.smithi150.stdout:5376 writesame 0x5ed0404 thru 0x5ed28dd data_size 0x35(0x24da bytes) 2024-09-21T23:11:41.307 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:41.302+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:41.310 INFO:teuthology.orchestra.run.smithi150.stdout:5378 trunc from 0x5ed28de to 0x8b7dd91 2024-09-21T23:11:41.325 INFO:teuthology.orchestra.run.smithi150.stdout:5380 writesame 0x8af242e thru 0x8af64b8 data_size 0x1f(0x408b bytes) 2024-09-21T23:11:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:5381 punch from 0x46a5437 to 0x46b342d, (0xdff6 bytes) 2024-09-21T23:11:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:5382 punch from 0x9db089 to 0x9dbd02, (0xc79 bytes) 2024-09-21T23:11:41.334 INFO:teuthology.orchestra.run.smithi150.stdout:5383 punch from 0x750e53a to 0x7510a35, (0x24fb bytes) 2024-09-21T23:11:41.359 INFO:teuthology.orchestra.run.smithi150.stdout:5384 write 0xc65f44d thru 0xc65f75d (0x311 bytes) 2024-09-21T23:11:41.366 INFO:teuthology.orchestra.run.smithi150.stdout:5385 read 0x28f6979 thru 0x28f6a9c (0x124 bytes) 2024-09-21T23:11:41.366 INFO:teuthology.orchestra.run.smithi150.stdout:5386 read 0x6e48eb4 thru 0x6e4ea06 (0x5b53 bytes) 2024-09-21T23:11:41.371 INFO:teuthology.orchestra.run.smithi150.stdout:5388 trunc from 0xc65f75e to 0xacc2f22 2024-09-21T23:11:41.393 INFO:teuthology.orchestra.run.smithi150.stdout:5389 write 0x101b8bf thru 0x1024ded (0x952f bytes) 2024-09-21T23:11:41.402 INFO:teuthology.orchestra.run.smithi150.stdout:5391 write 0x397241a thru 0x397e80e (0xc3f5 bytes) 2024-09-21T23:11:41.408 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:41.401+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:41.412 INFO:teuthology.orchestra.run.smithi150.stdout:5392 write 0x6d9c452 thru 0x6d9cd28 (0x8d7 bytes) 2024-09-21T23:11:41.441 INFO:teuthology.orchestra.run.smithi150.stdout:5393 write 0xc579c2d thru 0xc5875ae (0xd982 bytes) 2024-09-21T23:11:41.451 INFO:teuthology.orchestra.run.smithi150.stdout:5394 punch from 0x27d3fda to 0x27d6a47, (0x2a6d bytes) 2024-09-21T23:11:41.451 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:41.457 INFO:teuthology.orchestra.run.smithi150.stdout:5397 trunc from 0xc5875af to 0xa0b06e5 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 40.070489242000001, 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.464 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.466 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.468 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71501 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7200768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600544177, 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069501725, 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71501 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.469 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860150+0000", 2024-09-21T23:11:41.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-07 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860152+0000", 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9870000000000002e-06 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860158+0000", 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860162+0000", 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9180000000000002e-06 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929487+0000", 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069324330000000003 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929580+0000", 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2839999999999999e-05 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929651+0000", 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1416999999999997e-05 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.473 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600524372999999, 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600501218000002, 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.478 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:11:41.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:11:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:11:41.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600479001, 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.482 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.60045839, 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.486 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:41.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:11:41.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600438766, 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.490 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:41.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.494 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600417829000001, 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:41.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:11:41.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:11:41.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.498 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600395866, 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:41.502 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600376441999998, 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:11:41.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:11:41.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600357300999999, 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600330850999999, 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:11:41.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.514 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600311331, 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.515 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600288525, 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.519 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:41.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:11:41.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:11:41.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600268053000001, 2024-09-21T23:11:41.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:11:41.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:11:41.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.527 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600192287999999, 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.528 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:11:41.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.600168616000001, 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:11:41.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:11:41.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.595253445000001, 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:11:41.536 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:11:41.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.595228719000001, 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.540 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:41.544 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 19.595204144, 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:41.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:41.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.548 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.549 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:41.549 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:41.549 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:41.549 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:41.550 INFO:teuthology.orchestra.run.smithi150.stdout:5398 punch from 0x80e543e to 0x80e8a2e, (0x35f0 bytes) 2024-09-21T23:11:41.550 INFO:teuthology.orchestra.run.smithi150.stdout:5400 writesame 0x7d7946f thru 0x7d882ae data_size 0x20(0xee40 bytes) 2024-09-21T23:11:41.551 INFO:teuthology.orchestra.run.smithi150.stdout:5402 trunc from 0xa0b06e5 to 0x2ec6c7f 2024-09-21T23:11:41.551 INFO:teuthology.orchestra.run.smithi150.stdout:5403 read 0x27217c5 thru 0x2730f42 (0xf77e bytes) 2024-09-21T23:11:41.551 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:41.546+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:41.552 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:41.567 INFO:teuthology.orchestra.run.smithi150.stdout:5404 write 0x70877be thru 0x708d597 (0x5dda bytes) 2024-09-21T23:11:41.576 INFO:teuthology.orchestra.run.smithi150.stdout:5406 read 0x55bd6b9 thru 0x55c9b85 (0xc4cd bytes) 2024-09-21T23:11:41.592 INFO:teuthology.orchestra.run.smithi150.stdout:5407 write 0xa786c52 thru 0xa796ae4 (0xfe93 bytes) 2024-09-21T23:11:41.605 INFO:teuthology.orchestra.run.smithi150.stdout:5408 punch from 0x6596c82 to 0x65a4a41, (0xddbf bytes) 2024-09-21T23:11:41.621 INFO:teuthology.orchestra.run.smithi150.stdout:5409 writesame 0xe667b0d thru 0xe66f97d data_size 0x1(0x7e71 bytes) 2024-09-21T23:11:41.621 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:41.621 INFO:teuthology.orchestra.run.smithi150.stdout:5411 punch from 0x79d52a5 to 0x79d6e39, (0x1b94 bytes) 2024-09-21T23:11:41.621 INFO:teuthology.orchestra.run.smithi150.stdout:5414 punch from 0x2d20b to 0x36cdb, (0x9ad0 bytes) 2024-09-21T23:11:41.622 INFO:teuthology.orchestra.run.smithi150.stdout:5416 read 0xe4527d6 thru 0xe45bda6 (0x95d1 bytes) 2024-09-21T23:11:41.622 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:41.631 INFO:teuthology.orchestra.run.smithi150.stdout:5418 write 0xe9778c4 thru 0xe986afb (0xf238 bytes) 2024-09-21T23:11:41.650 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:41.646+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:41.656 INFO:teuthology.orchestra.run.smithi150.stdout:5419 trunc from 0xe986afc to 0x410abcc 2024-09-21T23:11:41.677 INFO:teuthology.orchestra.run.smithi150.stdout:5420 read 0x2220bed thru 0x2226f13 (0x6327 bytes) 2024-09-21T23:11:41.700 INFO:teuthology.orchestra.run.smithi150.stdout:5421 write 0xc6a5e06 thru 0xc6aedf7 (0x8ff2 bytes) 2024-09-21T23:11:41.710 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:41.710 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:41.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:11:41.723 INFO:teuthology.orchestra.run.smithi150.stdout:5423 trunc from 0xc6aedf8 to 0x2d33316 2024-09-21T23:11:41.735 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:41.735 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:41.740 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:41.742 INFO:teuthology.orchestra.run.smithi150.stdout:5424 punch from 0x1c2c892 to 0x1c33cf0, (0x745e bytes) 2024-09-21T23:11:41.742 INFO:teuthology.orchestra.run.smithi150.stdout:5425 write 0x137c4b3 thru 0x1383964 (0x74b2 bytes) 2024-09-21T23:11:41.750 INFO:teuthology.orchestra.run.smithi150.stdout:5426 read 0x30d53c thru 0x310218 (0x2cdd bytes) 2024-09-21T23:11:41.751 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:41.745+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:41.751 INFO:teuthology.orchestra.run.smithi150.stdout:5427 read 0xbf141d thru 0xbfb59d (0xa181 bytes) 2024-09-21T23:11:41.752 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:41.756 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:41.756 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:41.777 INFO:teuthology.orchestra.run.smithi150.stdout:5429 writesame 0xd726337 thru 0xd72f4b7 data_size 0x1(0x9181 bytes) 2024-09-21T23:11:41.778 INFO:teuthology.orchestra.run.smithi150.stdout:5430 write 0x4e34ad5 thru 0x4e35f7b (0x14a7 bytes) 2024-09-21T23:11:41.785 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:41.785 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:41.803 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:41.803 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:41.804 INFO:teuthology.orchestra.run.smithi150.stdout:5431 trunc from 0xd72f4b8 to 0x1b2c107 2024-09-21T23:11:41.827 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:41.828 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:41.847 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:41.847 INFO:teuthology.orchestra.run.smithi150.stdout:5433 punch from 0x62dcf0 to 0x636bbf, (0x8ecf bytes) 2024-09-21T23:11:41.847 INFO:teuthology.orchestra.run.smithi150.stdout:5435 read 0x544cbe thru 0x548604 (0x3947 bytes) 2024-09-21T23:11:41.849 INFO:teuthology.orchestra.run.smithi150.stdout:5436 trunc from 0x1b2c107 to 0x258ce3b 2024-09-21T23:11:41.851 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:41.846+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:41.852 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:41.852 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:41.873 INFO:teuthology.orchestra.run.smithi150.stdout:5437 trunc from 0x258ce3b to 0xb1db24f 2024-09-21T23:11:41.881 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:41.881 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:41.882 INFO:teuthology.orchestra.run.smithi150.stdout:5439 read 0x16e744f thru 0x16ef454 (0x8006 bytes) 2024-09-21T23:11:41.884 INFO:teuthology.orchestra.run.smithi150.stdout:5440 trunc from 0xb1db24f to 0xbe0afb6 2024-09-21T23:11:41.884 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:41.885 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:41.893 INFO:teuthology.orchestra.run.smithi150.stdout:5442 read 0xa7053fb thru 0xa705ff1 (0xbf7 bytes) 2024-09-21T23:11:41.893 INFO:teuthology.orchestra.run.smithi150.stdout:5444 read 0x19b9f6b thru 0x19c4b1a (0xabb0 bytes) 2024-09-21T23:11:41.893 INFO:teuthology.orchestra.run.smithi150.stdout:5445 read 0x5a63172 thru 0x5a6abe8 (0x7a77 bytes) 2024-09-21T23:11:41.893 INFO:teuthology.orchestra.run.smithi150.stdout:5448 write 0x8c26d22 thru 0x8c303ef (0x96ce bytes) 2024-09-21T23:11:41.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:41.903 INFO:teuthology.orchestra.run.smithi150.stdout:5449 write 0x86971f4 thru 0x86a0afe (0x990b bytes) 2024-09-21T23:11:41.909 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:41.910 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:41.915 INFO:teuthology.orchestra.run.smithi150.stdout:5450 read 0x970e7c thru 0x9778c6 (0x6a4b bytes) 2024-09-21T23:11:41.925 INFO:teuthology.orchestra.run.smithi150.stdout:5453 trunc from 0xbe0afb6 to 0x5754ea9 2024-09-21T23:11:41.952 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:41.949+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:41.963 INFO:teuthology.orchestra.run.smithi150.stdout:5454 write 0xb04640b thru 0xb04c331 (0x5f27 bytes) 2024-09-21T23:11:41.972 INFO:teuthology.orchestra.run.smithi150.stdout:5455 read 0x2516c66 thru 0x25181cb (0x1566 bytes) 2024-09-21T23:11:41.972 INFO:teuthology.orchestra.run.smithi150.stdout:5456 read 0x1b78940 thru 0x1b7da27 (0x50e8 bytes) 2024-09-21T23:11:41.972 INFO:teuthology.orchestra.run.smithi150.stdout:5457 read 0x5a8d45 thru 0x5b766b (0xe927 bytes) 2024-09-21T23:11:41.973 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:41.983 INFO:teuthology.orchestra.run.smithi150.stdout:5459 write 0xc1b0dc0 thru 0xc1b89d9 (0x7c1a bytes) 2024-09-21T23:11:41.991 INFO:teuthology.orchestra.run.smithi150.stdout:5460 read 0xb397e6f thru 0xb3a08ef (0x8a81 bytes) 2024-09-21T23:11:42.002 INFO:teuthology.orchestra.run.smithi150.stdout:5461 write 0xdb384a8 thru 0xdb3f924 (0x747d bytes) 2024-09-21T23:11:42.007 INFO:teuthology.orchestra.run.smithi150.stdout:5462 writesame 0xbaefd66 thru 0xbaf64da data_size 0x1(0x6775 bytes) 2024-09-21T23:11:42.031 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:42.031 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:42.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:42.053 INFO:teuthology.orchestra.run.smithi150.stdout:5463 write 0x43c99bd thru 0x43cd210 (0x3854 bytes) 2024-09-21T23:11:42.054 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:42.050+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:42.060 INFO:teuthology.orchestra.run.smithi150.stdout:5464 read 0x5435211 thru 0x543d118 (0x7f08 bytes) 2024-09-21T23:11:42.060 INFO:teuthology.orchestra.run.smithi150.stdout:5465 read 0xa55e03e thru 0xa5673b1 (0x9374 bytes) 2024-09-21T23:11:42.060 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:42.061 INFO:teuthology.orchestra.run.smithi150.stdout:5468 trunc from 0xdb3f925 to 0xd7a72ca 2024-09-21T23:11:42.078 INFO:teuthology.orchestra.run.smithi150.stdout:5470 read 0x4071780 thru 0x4071ec2 (0x743 bytes) 2024-09-21T23:11:42.078 INFO:teuthology.orchestra.run.smithi150.stdout:5472 read 0xc773be8 thru 0xc77c858 (0x8c71 bytes) 2024-09-21T23:11:42.078 INFO:teuthology.orchestra.run.smithi150.stdout:5474 writesame 0x8c0b6cb thru 0x8c0cdc9 data_size 0x7(0x16ff bytes) 2024-09-21T23:11:42.084 INFO:teuthology.orchestra.run.smithi150.stdout:5476 write 0xbb4d24a thru 0xbb56fcd (0x9d84 bytes) 2024-09-21T23:11:42.098 INFO:teuthology.orchestra.run.smithi150.stdout:5477 trunc from 0xd7a72ca to 0x740312d 2024-09-21T23:11:42.119 INFO:teuthology.orchestra.run.smithi150.stdout:5478 punch from 0x2db36de to 0x2db4ee2, (0x1804 bytes) 2024-09-21T23:11:42.119 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:42.119 INFO:teuthology.orchestra.run.smithi150.stdout:5480 write 0x578df58 thru 0x57917f2 (0x389b bytes) 2024-09-21T23:11:42.127 INFO:teuthology.orchestra.run.smithi150.stdout:5481 writesame 0x3de7c88 thru 0x3debef9 data_size 0x15(0x4272 bytes) 2024-09-21T23:11:42.136 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:42.136 INFO:teuthology.orchestra.run.smithi150.stdout:5484 write 0x2e7f421 thru 0x2e8e1a3 (0xed83 bytes) 2024-09-21T23:11:42.153 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:42.150+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:42.156 INFO:teuthology.orchestra.run.smithi150.stdout:5485 writesame 0x783bcdb thru 0x7844f7a data_size 0x33(0x92a0 bytes) 2024-09-21T23:11:42.156 INFO:teuthology.orchestra.run.smithi150.stdout:5487 punch from 0x45d13e4 to 0x45d3502, (0x211e bytes) 2024-09-21T23:11:42.156 INFO:teuthology.orchestra.run.smithi150.stdout:5488 punch from 0x3628ef to 0x371a3b, (0xf14c bytes) 2024-09-21T23:11:42.165 INFO:teuthology.orchestra.run.smithi150.stdout:5489 write 0x792f432 thru 0x7930faa (0x1b79 bytes) 2024-09-21T23:11:42.174 INFO:teuthology.orchestra.run.smithi150.stdout:5491 clone 49 order 25 su 262144 sc 14 2024-09-21T23:11:42.176 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:42.176 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:42.190 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:42.281 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:42.278+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:42.354 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:42.354 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:42.355 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:42.349+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:42.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:42.456 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:42.453+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:42.532 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:42.532 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:42.546 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:42.593 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:42.590+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:42.657 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:42.654+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:42.669 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:42.669 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:42.685 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:11:42.699 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:42.699 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:11:42.758 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:42.753+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:42.819 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:42.819 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:42.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:42.880 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:42.878+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:42.955 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:42.955 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:42.960 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:42.953+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:42.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:42.982 INFO:teuthology.orchestra.run.smithi150.stdout:leaving image image_client.0-clone48 intact 2024-09-21T23:11:43.060 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:43.058+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:43.101 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:43.101 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:43.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:43.162 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:43.157+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:43.262 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:43.257+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:43.295 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:43.295 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:43.311 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:43.363 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:43.357+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:43.437 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:43.437 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:43.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:43.463 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:43.458+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:43.563 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-21T23:11:43.564 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:43.558+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:43.579 INFO:teuthology.orchestra.run.smithi150.stdout:5492 punch from 0xbf623b to 0xbf72ff, (0x10c4 bytes) 2024-09-21T23:11:43.581 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:43.581 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:43.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:43.611 INFO:teuthology.orchestra.run.smithi150.stdout:5493 writesame 0xd0c2ba1 thru 0xd0c800c data_size 0x6(0x546c bytes) 2024-09-21T23:11:43.647 INFO:teuthology.orchestra.run.smithi150.stdout:5494 trunc from 0xd0c800d to 0x6ff87e0 2024-09-21T23:11:43.665 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:43.661+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:43.704 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:11:43.736 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:11:43.751 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:43.751 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:43.757 INFO:teuthology.orchestra.run.smithi150.stdout:5495 writesame 0x9e553f4 thru 0x9e5d2f0 data_size 0x1d(0x7efd bytes) 2024-09-21T23:11:43.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:43.767 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:43.761+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:43.768 INFO:teuthology.orchestra.run.smithi150.stdout:5496 write 0xbd3cbf8 thru 0xbd47a03 (0xae0c bytes) 2024-09-21T23:11:43.770 INFO:teuthology.orchestra.run.smithi150.stdout:5497 write 0xa1c45e6 thru 0xa1c6898 (0x22b3 bytes) 2024-09-21T23:11:43.778 INFO:teuthology.orchestra.run.smithi150.stdout:5498 writesame 0xbd9e115 thru 0xbda5344 data_size 0x3f(0x7230 bytes) 2024-09-21T23:11:43.779 INFO:teuthology.orchestra.run.smithi150.stdout:5499 write 0x8672e82 thru 0x867809e (0x521d bytes) 2024-09-21T23:11:43.781 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:43.781 INFO:teuthology.orchestra.run.smithi150.stdout:5503 write 0x93c8184 thru 0x93ca3ac (0x2229 bytes) 2024-09-21T23:11:43.782 INFO:teuthology.orchestra.run.smithi150.stdout:5504 write 0x854cb4b thru 0x8554af4 (0x7faa bytes) 2024-09-21T23:11:43.784 INFO:teuthology.orchestra.run.smithi150.stdout:5505 write 0x95f5ec thru 0x96f5db (0xfff0 bytes) 2024-09-21T23:11:43.786 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:43.787 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:43.787 INFO:teuthology.orchestra.run.smithi150.stdout:5509 trunc from 0xbda5345 to 0xc595744 2024-09-21T23:11:43.794 INFO:teuthology.orchestra.run.smithi150.stdout:5510 writesame 0x79062ba thru 0x7913f6e data_size 0x1(0xdcb5 bytes) 2024-09-21T23:11:43.826 INFO:teuthology.orchestra.run.smithi150.stdout:5511 writesame 0xb87ab38 thru 0xb87af0c data_size 0x3(0x3d5 bytes) 2024-09-21T23:11:43.828 INFO:teuthology.orchestra.run.smithi150.stdout:5512 punch from 0x3913c40 to 0x3917db3, (0x4173 bytes) 2024-09-21T23:11:43.828 INFO:teuthology.orchestra.run.smithi150.stdout:5514 punch from 0x83ff6a6 to 0x840a2fb, (0xac55 bytes) 2024-09-21T23:11:43.828 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:43.828 INFO:teuthology.orchestra.run.smithi150.stdout:5517 writesame 0x6602efc thru 0x6607c6f data_size 0x2(0x4d74 bytes) 2024-09-21T23:11:43.834 INFO:teuthology.orchestra.run.smithi150.stdout:5519 read 0x216870d thru 0x2172f40 (0xa834 bytes) 2024-09-21T23:11:43.835 INFO:teuthology.orchestra.run.smithi150.stdout:5520 clone 50 order 18 su 4096 sc 15 2024-09-21T23:11:43.866 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:43.861+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:43.927 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:43.927 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:43.937 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:43.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:43.980 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:43.978+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:44.068 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:44.061+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:44.082 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:44.082 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:44.099 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:44.099 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:44.108 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:44.108 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:44.115 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:44.115 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:44.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:44.132 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:44.132 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:44.150 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:44.151 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:44.171 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:44.171 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:44.191 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:44.191 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:44.210 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:44.210 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:44.212 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:44.209+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:44.269 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:44.266+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:44.286 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:44.286 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:44.286 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:44.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:44.370 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:44.366+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:44.471 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:44.466+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.741957768999995, 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:44.473 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:44.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:11:44.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.479 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094644276, 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.483 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094611601, 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:11:44.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:44.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094588265999999, 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.488 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.492 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094555701000001, 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:44.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:11:44.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094468078999999, 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:11:44.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:11:44.500 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094349332, 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094324567000001, 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.505 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:44.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.509 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094303419999999, 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.510 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:11:44.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.514 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094245723, 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:11:44.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:44.519 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094210841999999, 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.523 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094185317000001, 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:11:44.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094114131000001, 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:44.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.532 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094078284999998, 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.533 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094061192000002, 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:11:44.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.094045180999998, 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.542 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:44.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:11:44.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.547 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.093987933000001, 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.548 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:11:44.552 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.093906568000001, 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:11:44.553 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:11:44.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:11:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.093875017999999, 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:11:44.558 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:11:44.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:11:44.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.093728121000002, 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.566 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:44.567 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:44.567 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:44.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:44.612 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:44.609+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:44.673 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:44.670+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:44.696 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:44.696 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:44.696 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:44.696 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:44.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:44.712 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:11:44.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:11:44.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:11:44.781 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:44.778+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:44.874 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:44.870+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:44.876 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:44.876 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:44.876 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:44.876 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:44.876 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:44.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:44.948 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone49 from 0xc595744 (overlap 0x6ff87e0) to 0x0 2024-09-21T23:11:44.975 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:44.970+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:45.026 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.839293162000004, 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:11:45.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:45.032 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.839190075999994, 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:11:45.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:11:45.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.839122727000003, 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.037 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:45.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:11:45.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763935790999994, 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.045 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763880022999999, 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:11:45.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:11:45.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:11:45.049 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763848483999993, 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.050 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763815734999994, 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.054 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.058 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763788841999997, 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.063 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763763569000005, 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.068 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763732442000006, 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.069 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.763089085999994, 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:11:45.074 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:11:45.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757903158999994, 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:11:45.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:11:45.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:45.084 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757881675999997, 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:11:45.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757857389999998, 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:11:45.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:45.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:11:45.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:11:45.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757839723000004, 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.093 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:11:45.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:11:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757816137999995, 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:11:45.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:11:45.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757800122999996, 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:11:45.103 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:11:45.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757786292000006, 2024-09-21T23:11:45.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.109 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:11:45.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757754930999994, 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:11:45.113 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:11:45.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:11:45.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.117 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.757686508999996, 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.121 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.122 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.122 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:45.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:45.123 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:45.117+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:45.224 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:45.221+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:45.299 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:45.299 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:45.299 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:45.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:45.356 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:45.353+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:45.426 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:45.421+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:45.431 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-21T23:11:45.481 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:45.481 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:45.481 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:45.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:45.527 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:45.522+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:45.627 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:45.622+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 44.270839510999998, 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:11:45.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:11:45.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71501 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7200768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800894446000001, 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069501725, 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71501 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860150+0000", 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-07 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860152+0000", 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9870000000000002e-06 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860158+0000", 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860162+0000", 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9180000000000002e-06 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929487+0000", 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069324330000000003 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929580+0000", 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2839999999999999e-05 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929651+0000", 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1416999999999997e-05 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.673 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800874642, 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:11:45.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:11:45.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:11:45.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:11:45.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:11:45.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:11:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800851486999999, 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:11:45.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.683 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800829270000001, 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:11:45.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800808659000001, 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:11:45.688 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:11:45.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800789035000001, 2024-09-21T23:11:45.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:11:45.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:45.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800768097999999, 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.700 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:11:45.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800746135000001, 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:11:45.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:11:45.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800726710999999, 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:45.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:11:45.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:11:45.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.80070757, 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:11:45.713 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:11:45.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.80068112, 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:45.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:11:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800661600000002, 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.723 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:11:45.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:11:45.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:11:45.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800638794000001, 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.728 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:45.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:11:45.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:11:45.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:11:45.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.733 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800618321999998, 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:11:45.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:11:45.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:11:45.737 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800542557, 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.738 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.742 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.800518884999999, 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.746 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.795603713999999, 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:11:45.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:45.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:45.751 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.795578987999999, 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:11:45.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 23.795554413000001, 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:45.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:11:45.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:11:45.761 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.761 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.761 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.761 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:45.761 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:45.761 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:45.762 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:11:45.763 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:11:45.764 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:45.757+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:45.764 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:45.765 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:11:45.787 INFO:teuthology.orchestra.run.smithi150.stdout:5521 write 0x6744f3a thru 0x674aeb3 (0x5f7a bytes) 2024-09-21T23:11:45.842 INFO:teuthology.orchestra.run.smithi150.stdout:5522 writesame 0xaaf72d5 thru 0xab00ce5 data_size 0x1(0x9a11 bytes) 2024-09-21T23:11:45.863 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:45.858+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:45.893 INFO:teuthology.orchestra.run.smithi150.stdout:5524 write 0xb713194 thru 0xb71469b (0x1508 bytes) 2024-09-21T23:11:45.904 INFO:teuthology.orchestra.run.smithi150.stdout:5526 write 0xb2e44bb thru 0xb2ecdb8 (0x88fe bytes) 2024-09-21T23:11:45.945 INFO:teuthology.orchestra.run.smithi150.stdout:5527 read 0xa6d82dc thru 0xa6e52d9 (0xcffe bytes) 2024-09-21T23:11:45.947 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:45.948 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:45.948 INFO:teuthology.orchestra.run.smithi150.stdout:5528 write 0x4d8185e thru 0x4d8def8 (0xc69b bytes) 2024-09-21T23:11:45.964 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:45.961+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:45.999 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:46.002 INFO:teuthology.orchestra.run.smithi150.stdout:5529 write 0x88b7c5a thru 0x88c58db (0xdc82 bytes) 2024-09-21T23:11:46.062 INFO:teuthology.orchestra.run.smithi150.stdout:5530 punch from 0x121a54f to 0x1223976, (0x9427 bytes) 2024-09-21T23:11:46.062 INFO:teuthology.orchestra.run.smithi150.stdout:5532 read 0xa255eda thru 0xa259096 (0x31bd bytes) 2024-09-21T23:11:46.065 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:46.061+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:46.068 INFO:teuthology.orchestra.run.smithi150.stdout:5533 trunc from 0xc595744 to 0x81a4aaa 2024-09-21T23:11:46.154 INFO:teuthology.orchestra.run.smithi150.stdout:5534 trunc from 0x81a4aaa to 0xa29f0a4 2024-09-21T23:11:46.159 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:46.159 INFO:teuthology.orchestra.run.smithi150.stdout:5538 punch from 0x31c2f69 to 0x31c4e3b, (0x1ed2 bytes) 2024-09-21T23:11:46.159 INFO:teuthology.orchestra.run.smithi150.stdout:5539 read 0x4e2d4ef thru 0x4e34be9 (0x76fb bytes) 2024-09-21T23:11:46.167 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:46.162+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:46.167 INFO:teuthology.orchestra.run.smithi150.stdout:5540 trunc from 0xa29f0a4 to 0x446284f 2024-09-21T23:11:46.167 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:46.167 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:46.187 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:46.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:46.244 INFO:teuthology.orchestra.run.smithi150.stdout:5541 write 0xfd8118 thru 0xfe0d78 (0x8c61 bytes) 2024-09-21T23:11:46.285 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:46.282+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:46.296 INFO:teuthology.orchestra.run.smithi150.stdout:5542 write 0x4645325 thru 0x4645ba0 (0x87c bytes) 2024-09-21T23:11:46.298 INFO:teuthology.orchestra.run.smithi150.stdout:5543 read 0x2605be5 thru 0x260ac56 (0x5072 bytes) 2024-09-21T23:11:46.315 INFO:teuthology.orchestra.run.smithi150.stdout:5545 write 0xb8e69b4 thru 0xb8f419b (0xd7e8 bytes) 2024-09-21T23:11:46.334 INFO:teuthology.orchestra.run.smithi150.stdout:5546 punch from 0x3a26a1d to 0x3a30716, (0x9cf9 bytes) 2024-09-21T23:11:46.334 INFO:teuthology.orchestra.run.smithi150.stdout:5547 read 0xb49b02b thru 0xb4a607f (0xb055 bytes) 2024-09-21T23:11:46.334 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:46.334 INFO:teuthology.orchestra.run.smithi150.stdout:5549 writesame 0x71adb06 thru 0x71bb196 data_size 0x3b(0xd691 bytes) 2024-09-21T23:11:46.352 INFO:teuthology.orchestra.run.smithi150.stdout:5552 writesame 0xa5f4f9e thru 0xa5f7263 data_size 0x1(0x22c6 bytes) 2024-09-21T23:11:46.364 INFO:teuthology.orchestra.run.smithi150.stdout:5553 trunc from 0xb8f419c to 0xaa750a9 2024-09-21T23:11:46.368 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:46.362+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:46.374 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.769906941000002, 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:11:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.769860274000003, 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.769786207999999, 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:46.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:11:46.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.389 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.769723482000003, 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.769671531999997, 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:11:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.769458446000002, 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.767574420999999, 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:11:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:11:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:11:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.767526371000002, 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.767478064000002, 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:11:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:11:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.767430849, 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:11:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.767366619999997, 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.767314710000001, 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:11:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.762712415999999, 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.762657601000001, 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:11:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:11:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.762609877000003, 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:11:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:11:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:11:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.762542988, 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:11:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.758997710000003, 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:11:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:11:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.758950968999997, 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:11:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:11:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.758904446999999, 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:11:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.964208268999997, 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:46.468 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:46.470 INFO:teuthology.orchestra.run.smithi150.stdout:5554 read 0x5d925f4 thru 0x5d9915f (0x6b6c bytes) 2024-09-21T23:11:46.470 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:46.470 INFO:teuthology.orchestra.run.smithi150.stdout:5558 writesame 0xa4d5eaa thru 0xa4d9082 data_size 0x1(0x31d9 bytes) 2024-09-21T23:11:46.470 INFO:teuthology.orchestra.run.smithi150.stdout:5560 read 0x7be7678 thru 0x7bf30d6 (0xba5f bytes) 2024-09-21T23:11:46.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:46.471 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:46.471 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:46.472 INFO:teuthology.orchestra.run.smithi150.stdout:5561 write 0xe126b5d thru 0xe12ae0d (0x42b1 bytes) 2024-09-21T23:11:46.472 INFO:teuthology.orchestra.run.smithi150.stdout:5564 punch from 0x5a753fc to 0x5a849fb, (0xf5ff bytes) 2024-09-21T23:11:46.472 INFO:teuthology.orchestra.run.smithi150.stdout:5565 read 0xec832e thru 0xed51d4 (0xcea7 bytes) 2024-09-21T23:11:46.472 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:46.472 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:46.473 INFO:teuthology.orchestra.run.smithi150.stdout:5566 trunc from 0xe12ae0e to 0x2764cf8 2024-09-21T23:11:46.473 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:46.473 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:46.474 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:46.474 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:46.474 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:46.466+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:46.494 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:46.494 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:46.519 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:46.519 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:46.536 INFO:teuthology.orchestra.run.smithi150.stdout:5568 write 0xdf76733 thru 0xdf83238 (0xcb06 bytes) 2024-09-21T23:11:46.546 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:46.546 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:46.553 INFO:teuthology.orchestra.run.smithi150.stdout:5570 punch from 0xd63cac4 to 0xd643e4f, (0x738b bytes) 2024-09-21T23:11:46.553 INFO:teuthology.orchestra.run.smithi150.stdout:5572 write 0xcf6baeb thru 0xcf723ef (0x6905 bytes) 2024-09-21T23:11:46.563 INFO:teuthology.orchestra.run.smithi150.stdout:5573 read 0x23f0919 thru 0x23f3d9c (0x3484 bytes) 2024-09-21T23:11:46.565 INFO:teuthology.orchestra.run.smithi150.stdout:5575 read 0x79bfa47 thru 0x79c78b4 (0x7e6e bytes) 2024-09-21T23:11:46.565 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:46.565 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:46.565 INFO:teuthology.orchestra.run.smithi150.stdout:5583 write 0x9842a6e thru 0x98434f5 (0xa88 bytes) 2024-09-21T23:11:46.569 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:46.570 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:46.570 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:46.570 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:46.566+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:46.578 INFO:teuthology.orchestra.run.smithi150.stdout:5586 trunc from 0xdf83239 to 0x6fe46a2 2024-09-21T23:11:46.597 INFO:teuthology.orchestra.run.smithi150.stdout:5587 read 0x4145dcf thru 0x414b367 (0x5599 bytes) 2024-09-21T23:11:46.597 INFO:teuthology.orchestra.run.smithi150.stdout:5588 write 0xa32ca8 thru 0xa359a1 (0x2cfa bytes) 2024-09-21T23:11:46.632 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:46.632 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:46.638 INFO:teuthology.orchestra.run.smithi150.stdout:5590 write 0xb1c1343 thru 0xb1c6c9c (0x595a bytes) 2024-09-21T23:11:46.647 INFO:teuthology.orchestra.run.smithi150.stdout:5591 writesame 0x1bfc163 thru 0x1bfe958 data_size 0x42(0x27f6 bytes) 2024-09-21T23:11:46.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:46.666 INFO:teuthology.orchestra.run.smithi150.stdout:5592 write 0x2cd6aa3 thru 0x2cd89d9 (0x1f37 bytes) 2024-09-21T23:11:46.671 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:46.665+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:46.672 INFO:teuthology.orchestra.run.smithi150.stdout:5593 write 0xa72b8ae thru 0xa737fb3 (0xc706 bytes) 2024-09-21T23:11:46.689 INFO:teuthology.orchestra.run.smithi150.stdout:5594 write 0x25d8f61 thru 0x25e0459 (0x74f9 bytes) 2024-09-21T23:11:46.694 INFO:teuthology.orchestra.run.smithi150.stdout:5595 write 0xa1d70f3 thru 0xa1dfc31 (0x8b3f bytes) 2024-09-21T23:11:46.707 INFO:teuthology.orchestra.run.smithi150.stdout:5596 punch from 0x9c0637a to 0x9c0b555, (0x51db bytes) 2024-09-21T23:11:46.707 INFO:teuthology.orchestra.run.smithi150.stdout:5597 punch from 0x11c5e4c to 0x11d0b34, (0xace8 bytes) 2024-09-21T23:11:46.707 INFO:teuthology.orchestra.run.smithi150.stdout:5598 read 0x718423d thru 0x718ff81 (0xbd45 bytes) 2024-09-21T23:11:46.707 INFO:teuthology.orchestra.run.smithi150.stdout:5601 writesame 0x1fee8f9 thru 0x1ff82ff data_size 0x7(0x9a07 bytes) 2024-09-21T23:11:46.726 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:11:46.748 INFO:teuthology.orchestra.run.smithi150.stdout:5602 clone 51 order 21 su 1048576 sc 13 2024-09-21T23:11:46.757 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:11:46.772 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:46.769+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:46.796 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:46.796 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:46.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:46.873 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:46.869+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:46.968 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:46.968 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:46.974 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:46.969+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:46.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:47.075 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:47.070+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:47.091 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:47.091 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:47.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:47.176 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:47.173+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:47.265 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:47.265 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:47.277 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:47.273+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:47.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:47.378 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:47.373+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:47.394 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:47.394 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:47.407 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:47.478 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:47.474+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:47.523 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:47.523 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:47.536 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:47.579 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:47.573+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:47.680 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:47.677+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:47.697 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:47.697 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:47.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:47.739 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:11:47.763 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:11:47.766 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:11:47.781 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:47.777+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:47.842 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:47.842 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:47.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:47.900 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:47.898+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:47.982 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:47.978+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:47.993 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone50 from 0xb1c6c9d (overlap 0x2764cf8) to 0x0 2024-09-21T23:11:47.999 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:47.999 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:48.020 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:48.083 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:48.077+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:48.184 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:48.178+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:48.192 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:48.192 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:48.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:48.300 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:48.297+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:48.386 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:48.382+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:48.433 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:48.433 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:48.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:48.500 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:48.497+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:48.588 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:48.581+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:48.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:48.620 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:48.620 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:48.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:48.714 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-21T23:11:48.716 INFO:teuthology.orchestra.run.smithi150.stdout:5603 punch from 0x1504fd6 to 0x1508808, (0x3832 bytes) 2024-09-21T23:11:48.720 INFO:teuthology.orchestra.run.smithi150.stdout:5604 read 0xefe53 thru 0xfa456 (0xa604 bytes) 2024-09-21T23:11:48.720 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:48.717+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:48.722 INFO:teuthology.orchestra.run.smithi150.stdout:5605 read 0x4c4ffaa thru 0x4c5bf0d (0xbf64 bytes) 2024-09-21T23:11:48.722 INFO:teuthology.orchestra.run.smithi150.stdout:5606 read 0x39a961f thru 0x39af92a (0x630c bytes) 2024-09-21T23:11:48.725 INFO:teuthology.orchestra.run.smithi150.stdout:5607 trunc from 0xb1c6c9d to 0xd7b3d3d 2024-09-21T23:11:48.731 INFO:teuthology.orchestra.run.smithi150.stdout:5608 write 0x4ac9e86 thru 0x4acf951 (0x5acc bytes) 2024-09-21T23:11:48.747 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:11:48.751 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:48.751 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:48.751 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:11:48.753 INFO:teuthology.orchestra.run.smithi150.stdout:5609 trunc from 0xd7b3d3d to 0x8a7895 2024-09-21T23:11:48.781 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:48.782 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:48.785 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T23:11:48.789 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:48.785+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:48.820 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:48.820 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:48.824 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:48.824 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:48.834 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:48.854 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:48.855 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:48.878 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:48.878 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:48.889 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:48.886+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:48.895 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:48.895 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:48.912 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:48.913 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:48.929 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:48.929 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:48.990 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:48.985+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.276400527999996, 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.007 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:11:49.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.629087035000001, 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.014 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.629054360000001, 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.018 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:49.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.022 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.629031025, 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.026 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628998459999998, 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:11:49.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:11:49.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628910837999999, 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.031 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628792091000001, 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628767325999998, 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:11:49.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:11:49.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628746179, 2024-09-21T23:11:49.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.048 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628688482000001, 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.049 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:11:49.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:11:49.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:49.053 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628653601, 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:49.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:11:49.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.057 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628628075999998, 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.058 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:11:49.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.062 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628556889999999, 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:11:49.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628521043999999, 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.067 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:11:49.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:11:49.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628503950999999, 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.073 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.077 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628487939999999, 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:11:49.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:49.082 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628430691999998, 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.086 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628349326999999, 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.087 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:11:49.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628317776999999, 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:11:49.091 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:11:49.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:11:49.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.628170879999999, 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:11:49.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:11:49.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:11:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:11:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.100 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.100 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:49.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:49.102 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:49.098+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:49.111 INFO:teuthology.orchestra.run.smithi150.stdout:5611 write 0xeb3f3db thru 0xeb4c3ab (0xcfd1 bytes) 2024-09-21T23:11:49.119 INFO:teuthology.orchestra.run.smithi150.stdout:5612 write 0x67074b4 thru 0x6716f29 (0xfa76 bytes) 2024-09-21T23:11:49.126 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.126 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.126 INFO:teuthology.orchestra.run.smithi150.stdout:5616 read 0xd46b1f8 thru 0xd477adc (0xc8e5 bytes) 2024-09-21T23:11:49.126 INFO:teuthology.orchestra.run.smithi150.stdout:5617 write 0xc5a64bc thru 0xc5b2c2c (0xc771 bytes) 2024-09-21T23:11:49.132 INFO:teuthology.orchestra.run.smithi150.stdout:5618 read 0x6778451 thru 0x677ed5e (0x690e bytes) 2024-09-21T23:11:49.149 INFO:teuthology.orchestra.run.smithi150.stdout:5620 trunc from 0xeb4c3ac to 0x6c8ac3e 2024-09-21T23:11:49.187 INFO:teuthology.orchestra.run.smithi150.stdout:5621 write 0x9b2d840 thru 0x9b2d86b (0x2c bytes) 2024-09-21T23:11:49.193 INFO:teuthology.orchestra.run.smithi150.stdout:5622 write 0x14b2946 thru 0x14b2a47 (0x102 bytes) 2024-09-21T23:11:49.196 INFO:teuthology.orchestra.run.smithi150.stdout:5623 write 0x68863fc thru 0x688d4b1 (0x70b6 bytes) 2024-09-21T23:11:49.202 INFO:teuthology.orchestra.run.smithi150.stdout:5624 punch from 0x984221f to 0x98425fe, (0x3df bytes) 2024-09-21T23:11:49.202 INFO:teuthology.orchestra.run.smithi150.stdout:5625 write 0x5d897a1 thru 0x5d8b11f (0x197f bytes) 2024-09-21T23:11:49.202 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:49.197+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:49.218 INFO:teuthology.orchestra.run.smithi150.stdout:5626 writesame 0x9cd9210 thru 0x9ce7809 data_size 0x1(0xe5fa bytes) 2024-09-21T23:11:49.219 INFO:teuthology.orchestra.run.smithi150.stdout:5627 read 0x4d22ddd thru 0x4d28b26 (0x5d4a bytes) 2024-09-21T23:11:49.231 INFO:teuthology.orchestra.run.smithi150.stdout:5628 trunc from 0x9ce780a to 0x49a582d 2024-09-21T23:11:49.272 INFO:teuthology.orchestra.run.smithi150.stdout:5629 trunc from 0x49a582d to 0xbe03564 2024-09-21T23:11:49.274 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:49.274 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:49.274 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:49.274 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:49.287 INFO:teuthology.orchestra.run.smithi150.stdout:5632 read 0x6b458bd thru 0x6b4b9cd (0x6111 bytes) 2024-09-21T23:11:49.287 INFO:teuthology.orchestra.run.smithi150.stdout:5634 write 0x343f14d thru 0x34440af (0x4f63 bytes) 2024-09-21T23:11:49.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:49.293 INFO:teuthology.orchestra.run.smithi150.stdout:5636 read 0x2b068fb thru 0x2b0ca34 (0x613a bytes) 2024-09-21T23:11:49.313 INFO:teuthology.orchestra.run.smithi150.stdout:5637 trunc from 0xbe03564 to 0x4dbce91 2024-09-21T23:11:49.332 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:49.329+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:49.347 INFO:teuthology.orchestra.run.smithi150.stdout:5638 trunc from 0x4dbce91 to 0xcea19d2 2024-09-21T23:11:49.359 INFO:teuthology.orchestra.run.smithi150.stdout:5642 writesame 0x6c1845f thru 0x6c1ef80 data_size 0x15(0x6b22 bytes) 2024-09-21T23:11:49.368 INFO:teuthology.orchestra.run.smithi150.stdout:5643 read 0x9bd5653 thru 0x9be3d3e (0xe6ec bytes) 2024-09-21T23:11:49.368 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.368 INFO:teuthology.orchestra.run.smithi150.stdout:5646 punch from 0x4620f58 to 0x4624e61, (0x3f09 bytes) 2024-09-21T23:11:49.368 INFO:teuthology.orchestra.run.smithi150.stdout:5647 write 0x1882e03 thru 0x1886294 (0x3492 bytes) 2024-09-21T23:11:49.400 INFO:teuthology.orchestra.run.smithi150.stdout:5648 trunc from 0xcea19d2 to 0x26b8b88 2024-09-21T23:11:49.404 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:49.398+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:49.457 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:49.457 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:49.457 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:49.457 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:49.457 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:49.462 INFO:teuthology.orchestra.run.smithi150.stdout:5649 write 0xd78da95 thru 0xd7989e4 (0xaf50 bytes) 2024-09-21T23:11:49.470 INFO:teuthology.orchestra.run.smithi150.stdout:5650 writesame 0x29231c5 thru 0x292e1b4 data_size 0x28(0xaff0 bytes) 2024-09-21T23:11:49.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:49.477 INFO:teuthology.orchestra.run.smithi150.stdout:5652 read 0x825f629 thru 0x826d81e (0xe1f6 bytes) 2024-09-21T23:11:49.477 INFO:teuthology.orchestra.run.smithi150.stdout:5653 read 0x33dc3a3 thru 0x33e21f5 (0x5e53 bytes) 2024-09-21T23:11:49.477 INFO:teuthology.orchestra.run.smithi150.stdout:5655 writesame 0x2cb5ca9 thru 0x2cbb874 data_size 0x32(0x5bcc bytes) 2024-09-21T23:11:49.483 INFO:teuthology.orchestra.run.smithi150.stdout:5657 write 0xd22fce1 thru 0xd23a835 (0xab55 bytes) 2024-09-21T23:11:49.499 INFO:teuthology.orchestra.run.smithi150.stdout:5658 write 0xd88d439 thru 0xd891657 (0x421f bytes) 2024-09-21T23:11:49.504 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:49.498+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:49.505 INFO:teuthology.orchestra.run.smithi150.stdout:5659 read 0xf0d4c5 thru 0xf1d15f (0xfc9b bytes) 2024-09-21T23:11:49.506 INFO:teuthology.orchestra.run.smithi150.stdout:5660 read 0x3821049 thru 0x382883d (0x77f5 bytes) 2024-09-21T23:11:49.519 INFO:teuthology.orchestra.run.smithi150.stdout:5663 write 0xe4bbe50 thru 0xe4c6d9b (0xaf4c bytes) 2024-09-21T23:11:49.522 INFO:teuthology.orchestra.run.smithi150.stdout:5664 writesame 0x1cb7ab7 thru 0x1cb8f5d data_size 0x1(0x14a7 bytes) 2024-09-21T23:11:49.556 INFO:teuthology.orchestra.run.smithi150.stdout:5666 trunc from 0xe4c6d9c to 0x33cc026 2024-09-21T23:11:49.580 INFO:teuthology.orchestra.run.smithi150.stdout:5667 punch from 0x295e22f to 0x2965821, (0x75f2 bytes) 2024-09-21T23:11:49.580 INFO:teuthology.orchestra.run.smithi150.stdout:5668 punch from 0x1a3e888 to 0x1a46c6f, (0x83e7 bytes) 2024-09-21T23:11:49.581 INFO:teuthology.orchestra.run.smithi150.stdout:5670 write 0x40969a thru 0x413e69 (0xa7d0 bytes) 2024-09-21T23:11:49.584 INFO:teuthology.orchestra.run.smithi150.stdout:5671 read 0x31f5e20 thru 0x31fdd99 (0x7f7a bytes) 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:49.598 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.411144724, 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.602 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.411041638, 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:11:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:11:49.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.606 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.41097428899999, 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:11:49.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:11:49.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.610 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.335787353, 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:11:49.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.335731585, 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.615 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:11:49.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.335700046, 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:11:49.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:11:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.335667297, 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.335640404, 2024-09-21T23:11:49.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.335615131, 2024-09-21T23:11:49.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:49.636 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.335584004, 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:49.640 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.334940648, 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:49.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:11:49.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:11:49.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.644 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329754721, 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:11:49.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:11:49.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.648 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329733238, 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329708952, 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.656 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329691285, 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:11:49.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:11:49.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:11:49.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:11:49.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.3296677, 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:49.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:11:49.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:11:49.664 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329651685, 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:11:49.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:49.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:11:49.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:11:49.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329637854, 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.669 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329606493, 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.673 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:11:49.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.329538071, 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.681 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:49.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5672 writesame 0x5796151 thru 0x57a31f7 data_size 0xf(0xd0a7 bytes) 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5674 writesame 0x14bd4b thru 0x154672 data_size 0x58(0x8928 bytes) 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5676 writesame 0x4f0341d thru 0x4f0eaf6 data_size 0x1f(0xb6da bytes) 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5677 read 0x1c0a3da thru 0x1c19139 (0xed60 bytes) 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5678 punch from 0x3e9e03a to 0x3ea3b98, (0x5b5e bytes) 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5679 write 0x9d13e01 thru 0x9d14409 (0x609 bytes) 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5680 read 0x8f10092 thru 0x8f12c38 (0x2ba7 bytes) 2024-09-21T23:11:49.684 INFO:teuthology.orchestra.run.smithi150.stdout:5683 write 0x6764c06 thru 0x677085c (0xbc57 bytes) 2024-09-21T23:11:49.685 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.685 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.685 INFO:teuthology.orchestra.run.smithi150.stdout:5688 writesame 0x5e365ee thru 0x5e3cd35 data_size 0x14(0x6748 bytes) 2024-09-21T23:11:49.685 INFO:teuthology.orchestra.run.smithi150.stdout:5689 punch from 0x4167cab to 0x4177690, (0xf9e5 bytes) 2024-09-21T23:11:49.685 INFO:teuthology.orchestra.run.smithi150.stdout:5690 write 0x4785441 thru 0x478d582 (0x8142 bytes) 2024-09-21T23:11:49.685 INFO:teuthology.orchestra.run.smithi150.stdout:5691 trunc from 0x9d1440a to 0xc674d93 2024-09-21T23:11:49.685 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:49.678+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:49.693 INFO:teuthology.orchestra.run.smithi150.stdout:5694 write 0x1c09a7d thru 0x1c0cb48 (0x30cc bytes) 2024-09-21T23:11:49.706 INFO:teuthology.orchestra.run.smithi150.stdout:5695 trunc from 0xc674d93 to 0x8e0c059 2024-09-21T23:11:49.729 INFO:teuthology.orchestra.run.smithi150.stdout:5696 writesame 0x8c6a700 thru 0x8c6ef80 data_size 0x17(0x4881 bytes) 2024-09-21T23:11:49.736 INFO:teuthology.orchestra.run.smithi150.stdout:5698 punch from 0x5f136 to 0x6a59b, (0xb465 bytes) 2024-09-21T23:11:49.736 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.736 INFO:teuthology.orchestra.run.smithi150.stdout:5702 writesame 0x4ace68e thru 0x4ad8811 data_size 0x2(0xa184 bytes) 2024-09-21T23:11:49.755 INFO:teuthology.orchestra.run.smithi150.stdout:5703 write 0x2ba048c thru 0x2ba8900 (0x8475 bytes) 2024-09-21T23:11:49.763 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T23:11:49.763 INFO:teuthology.orchestra.run.smithi150.stdout:5704 writesame 0x28b04f1 thru 0x28b9290 data_size 0x2c(0x8da0 bytes) 2024-09-21T23:11:49.772 INFO:teuthology.orchestra.run.smithi150.stdout:5705 punch from 0x658cf6e to 0x659bc96, (0xed28 bytes) 2024-09-21T23:11:49.783 INFO:teuthology.orchestra.run.smithi150.stdout:5706 write 0x94fa1ee thru 0x95048fe (0xa711 bytes) 2024-09-21T23:11:49.783 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:49.777+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:49.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T23:11:49.797 INFO:teuthology.orchestra.run.smithi150.stdout:5707 writesame 0x9a4d513 thru 0x9a53a64 data_size 0x63(0x6552 bytes) 2024-09-21T23:11:49.813 INFO:teuthology.orchestra.run.smithi150.stdout:5709 write 0xeaed96f thru 0xeaf0a7f (0x3111 bytes) 2024-09-21T23:11:49.817 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:49.817 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:49.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.817 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:115970 3.1a 3:59600651:::rbd_data.114969d6b98.0000000000000075:head [set-alloc-hint object_size 2097152 write_size 2097152,write 973167~12561 in=12561b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:11:49.817 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:49.815852+0000", 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0016383179999999999, 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016782940000000001, 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 115970 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.815852+0000", 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.815852+0000", 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.815854+0000", 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0930000000000001e-06 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.815878+0000", 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.402e-05 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.815908+0000", 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9638e-05 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.815922+0000", 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4516e-05 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.815979+0000", 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6858999999999999e-05 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.816104+0000", 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012457899999999999 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1", 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.816219+0000", 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011535 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:11:49.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:49.817522+0000", 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00130294 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:49.822 INFO:teuthology.orchestra.run.smithi150.stdout:5710 write 0x3a296 thru 0x3e964 (0x46cf bytes) 2024-09-21T23:11:49.823 INFO:teuthology.orchestra.run.smithi150.stdout:5711 write 0x3b87cdc thru 0x3b8e37b (0x66a0 bytes) 2024-09-21T23:11:49.825 INFO:teuthology.orchestra.run.smithi150.stdout:5713 read 0x49d6c7d thru 0x49dab01 (0x3e85 bytes) 2024-09-21T23:11:49.826 INFO:teuthology.orchestra.run.smithi150.stdout:5714 read 0xb6b84b8 thru 0xb6be823 (0x636c bytes) 2024-09-21T23:11:49.826 INFO:teuthology.orchestra.run.smithi150.stdout:5715 read 0x1bb81b9 thru 0x1bb9199 (0xfe1 bytes) 2024-09-21T23:11:49.826 INFO:teuthology.orchestra.run.smithi150.stdout:5717 writesame 0x6704b37 thru 0x670676a data_size 0x13(0x1c34 bytes) 2024-09-21T23:11:49.828 INFO:teuthology.orchestra.run.smithi150.stdout:5719 write 0x166c0a1 thru 0x166cd14 (0xc74 bytes) 2024-09-21T23:11:49.832 INFO:teuthology.orchestra.run.smithi150.stdout:5720 write 0x474e657 thru 0x4754d9e (0x6748 bytes) 2024-09-21T23:11:49.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:49.834 INFO:teuthology.orchestra.run.smithi150.stdout:5721 write 0xda4dc9 thru 0xdb2148 (0xd380 bytes) 2024-09-21T23:11:49.836 INFO:teuthology.orchestra.run.smithi150.stdout:5723 write 0xa8555d0 thru 0xa863355 (0xdd86 bytes) 2024-09-21T23:11:49.856 INFO:teuthology.orchestra.run.smithi150.stdout:5725 trunc from 0xeaf0a80 to 0x49fc483 2024-09-21T23:11:49.885 INFO:teuthology.orchestra.run.smithi150.stdout:5726 write 0x96a65f8 thru 0x96a8c40 (0x2649 bytes) 2024-09-21T23:11:49.890 INFO:teuthology.orchestra.run.smithi150.stdout:5727 read 0x6d3479f thru 0x6d43bf1 (0xf453 bytes) 2024-09-21T23:11:49.890 INFO:teuthology.orchestra.run.smithi150.stdout:5728 read 0x31e9556 thru 0x31f2b33 (0x95de bytes) 2024-09-21T23:11:49.890 INFO:teuthology.orchestra.run.smithi150.stdout:5729 punch from 0x6b2b37 to 0x6b766f, (0x4b38 bytes) 2024-09-21T23:11:49.890 INFO:teuthology.orchestra.run.smithi150.stdout:5730 punch from 0x4c21009 to 0x4c27091, (0x6088 bytes) 2024-09-21T23:11:49.890 INFO:teuthology.orchestra.run.smithi150.stdout:5732 writesame 0x710a629 thru 0x71107b6 data_size 0x1(0x618e bytes) 2024-09-21T23:11:49.910 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.910 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.910 INFO:teuthology.orchestra.run.smithi150.stdout:5737 read 0x27e0468 thru 0x27e8714 (0x82ad bytes) 2024-09-21T23:11:49.910 INFO:teuthology.orchestra.run.smithi150.stdout:5738 read 0x7dca0a7 thru 0x7dcc177 (0x20d1 bytes) 2024-09-21T23:11:49.911 INFO:teuthology.orchestra.run.smithi150.stdout:5739 read 0x96a06cf thru 0x96a8c40 (0x8572 bytes) 2024-09-21T23:11:49.911 INFO:teuthology.orchestra.run.smithi150.stdout:5740 read 0x31b32c4 thru 0x31c042d (0xd16a bytes) 2024-09-21T23:11:49.911 INFO:teuthology.orchestra.run.smithi150.stdout:5742 read 0xcce46e thru 0xcdc717 (0xe2aa bytes) 2024-09-21T23:11:49.911 INFO:teuthology.orchestra.run.smithi150.stdout:5743 read 0x8e9f8fa thru 0x8ea21d6 (0x28dd bytes) 2024-09-21T23:11:49.911 INFO:teuthology.orchestra.run.smithi150.stdout:5745 writesame 0x54b85ae thru 0x54ba62c data_size 0x2f(0x207f bytes) 2024-09-21T23:11:49.920 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:49.917+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:49.926 INFO:teuthology.orchestra.run.smithi150.stdout:5746 trunc from 0x96a8c41 to 0x4b544c8 2024-09-21T23:11:49.939 INFO:teuthology.orchestra.run.smithi150.stdout:5747 read 0x1d29797 thru 0x1d32f38 (0x97a2 bytes) 2024-09-21T23:11:49.953 INFO:teuthology.orchestra.run.smithi150.stdout:5749 writesame 0x9383a0d thru 0x938668c data_size 0x20(0x2c80 bytes) 2024-09-21T23:11:49.953 INFO:teuthology.orchestra.run.smithi150.stdout:5750 punch from 0x9ae049 to 0x9b8648, (0xa5ff bytes) 2024-09-21T23:11:49.954 INFO:teuthology.orchestra.run.smithi150.stdout:5751 trunc from 0x938668d to 0x9fd4486 2024-09-21T23:11:49.961 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.961 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:49.961 INFO:teuthology.orchestra.run.smithi150.stdout:5755 read 0x4b1ca2f thru 0x4b297d8 (0xcdaa bytes) 2024-09-21T23:11:49.974 INFO:teuthology.orchestra.run.smithi150.stdout:5757 write 0xee1e9b3 thru 0xee27a5e (0x90ac bytes) 2024-09-21T23:11:49.979 INFO:teuthology.orchestra.run.smithi150.stdout:5758 read 0x805ebe8 thru 0x8069fcc (0xb3e5 bytes) 2024-09-21T23:11:49.979 INFO:teuthology.orchestra.run.smithi150.stdout:5760 write 0xe1450b3 thru 0xe14d42f (0x837d bytes) 2024-09-21T23:11:49.984 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:49.982+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:49.993 INFO:teuthology.orchestra.run.smithi150.stdout:5761 trunc from 0xee27a5f to 0x97eb984 2024-09-21T23:11:50.001 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:50.001 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:50.001 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:50.001 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:50.001 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:50.005 INFO:teuthology.orchestra.run.smithi150.stdout:5763 writesame 0x4a6b4a5 thru 0x4a79234 data_size 0x28(0xdd90 bytes) 2024-09-21T23:11:50.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:50.019 INFO:teuthology.orchestra.run.smithi150.stdout:5765 write 0xce7b581 thru 0xce8132b (0x5dab bytes) 2024-09-21T23:11:50.023 INFO:teuthology.orchestra.run.smithi150.stdout:5766 write 0x99449c2 thru 0x994c306 (0x7945 bytes) 2024-09-21T23:11:50.027 INFO:teuthology.orchestra.run.smithi150.stdout:5767 read 0xcb0f4b5 thru 0xcb11ffa (0x2b46 bytes) 2024-09-21T23:11:50.027 INFO:teuthology.orchestra.run.smithi150.stdout:5768 read 0x9c3e37a thru 0x9c41f1f (0x3ba6 bytes) 2024-09-21T23:11:50.027 INFO:teuthology.orchestra.run.smithi150.stdout:5769 punch from 0x3e58a91 to 0x3e59c85, (0x11f4 bytes) 2024-09-21T23:11:50.027 INFO:teuthology.orchestra.run.smithi150.stdout:5772 trunc from 0xce8132c to 0xcbd36e7 2024-09-21T23:11:50.044 INFO:teuthology.orchestra.run.smithi150.stdout:5774 trunc from 0xcbd36e7 to 0x612065f 2024-09-21T23:11:50.055 INFO:teuthology.orchestra.run.smithi150.stdout:5775 write 0x42f7aad thru 0x4305027 (0xd57b bytes) 2024-09-21T23:11:50.066 INFO:teuthology.orchestra.run.smithi150.stdout:5777 write 0x69dc0b1 thru 0x69de756 (0x26a6 bytes) 2024-09-21T23:11:50.070 INFO:teuthology.orchestra.run.smithi150.stdout:5778 read 0x533bf05 thru 0x534973b (0xd837 bytes) 2024-09-21T23:11:50.070 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.070 INFO:teuthology.orchestra.run.smithi150.stdout:5781 read 0x21f3614 thru 0x21f76be (0x40ab bytes) 2024-09-21T23:11:50.070 INFO:teuthology.orchestra.run.smithi150.stdout:5782 trunc from 0x69de757 to 0x6aa295a 2024-09-21T23:11:50.075 INFO:teuthology.orchestra.run.smithi150.stdout:5784 read 0x26af72a thru 0x26b35a3 (0x3e7a bytes) 2024-09-21T23:11:50.075 INFO:teuthology.orchestra.run.smithi150.stdout:5785 read 0x1985ddd thru 0x198916d (0x3391 bytes) 2024-09-21T23:11:50.084 INFO:teuthology.orchestra.run.smithi150.stdout:5786 writesame 0x7fafdb9 thru 0x7fb6653 data_size 0x1(0x689b bytes) 2024-09-21T23:11:50.084 INFO:teuthology.orchestra.run.smithi150.stdout:5787 write 0x6be081a thru 0x6be3cf6 (0x34dd bytes) 2024-09-21T23:11:50.095 INFO:teuthology.orchestra.run.smithi150.stdout:5788 writesame 0x96af800 thru 0x96b1d2a data_size 0xb(0x252b bytes) 2024-09-21T23:11:50.095 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.095 INFO:teuthology.orchestra.run.smithi150.stdout:5792 punch from 0x5c99739 to 0x5ca7007, (0xd8ce bytes) 2024-09-21T23:11:50.095 INFO:teuthology.orchestra.run.smithi150.stdout:5793 read 0x4a3d0f2 thru 0x4a4d02b (0xff3a bytes) 2024-09-21T23:11:50.096 INFO:teuthology.orchestra.run.smithi150.stdout:5794 read 0xe43301 thru 0xe4eaab (0xb7ab bytes) 2024-09-21T23:11:50.096 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.104 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:50.101+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:50.107 INFO:teuthology.orchestra.run.smithi150.stdout:5797 trunc from 0x96b1d2b to 0x25ec6c5 2024-09-21T23:11:50.124 INFO:teuthology.orchestra.run.smithi150.stdout:5799 punch from 0x206e91b to 0x2072c25, (0x430a bytes) 2024-09-21T23:11:50.130 INFO:teuthology.orchestra.run.smithi150.stdout:5800 write 0x270591c thru 0x270e0bc (0x87a1 bytes) 2024-09-21T23:11:50.146 INFO:teuthology.orchestra.run.smithi150.stdout:5801 trunc from 0x270e0bd to 0x8ef538a 2024-09-21T23:11:50.153 INFO:teuthology.orchestra.run.smithi150.stdout:5802 read 0x2747885 thru 0x2756144 (0xe8c0 bytes) 2024-09-21T23:11:50.162 INFO:teuthology.orchestra.run.smithi150.stdout:5803 trunc from 0x8ef538a to 0x3e4eb6e 2024-09-21T23:11:50.184 INFO:teuthology.orchestra.run.smithi150.stdout:5804 writesame 0x6b29ca4 thru 0x6b2cbf9 data_size 0x49(0x2f56 bytes) 2024-09-21T23:11:50.184 INFO:teuthology.orchestra.run.smithi150.stdout:5807 read 0xaf2714 thru 0xaf48bc (0x21a9 bytes) 2024-09-21T23:11:50.184 INFO:teuthology.orchestra.run.smithi150.stdout:5808 punch from 0x33d073 to 0x33ef4e, (0x1edb bytes) 2024-09-21T23:11:50.184 INFO:teuthology.orchestra.run.smithi150.stdout:5809 punch from 0x64a4341 to 0x64ae99e, (0xa65d bytes) 2024-09-21T23:11:50.184 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.184 INFO:teuthology.orchestra.run.smithi150.stdout:5812 write 0x4efdbfa thru 0x4f0603f (0x8446 bytes) 2024-09-21T23:11:50.185 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:50.181+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 48.794740138000002, 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.188 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71501 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7200768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324795073000001, 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069501725, 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.193 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71501 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860150+0000", 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-07 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860152+0000", 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9870000000000002e-06 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860158+0000", 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860162+0000", 2024-09-21T23:11:50.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9180000000000002e-06 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929487+0000", 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069324330000000003 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929580+0000", 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2839999999999999e-05 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929651+0000", 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1416999999999997e-05 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324775269, 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.198 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.202 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324752113999999, 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:11:50.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324729897000001, 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.207 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:11:50.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324709286000001, 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.211 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324689662000001, 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.215 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:50.219 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324668724999999, 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:50.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:11:50.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324646762, 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.224 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:50.228 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324627337999999, 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:50.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:11:50.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324608197, 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:11:50.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:11:50.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:50.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324581747, 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:11:50.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:11:50.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324562227000001, 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:11:50.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:11:50.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324539421000001, 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324518949000002, 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:11:50.250 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:11:50.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324443184, 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:11:50.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:11:50.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.324419511999999, 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:11:50.259 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:11:50.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.319504340999998, 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.264 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:11:50.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.319479614999999, 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:11:50.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:11:50.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 28.319455040000001, 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:11:50.272 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:50.276 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:50.277 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:50.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:50.279 INFO:teuthology.orchestra.run.smithi150.stdout:5813 read 0x675872a thru 0x67626bb (0x9f92 bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5814 trunc from 0x6b2cbfa to 0x375d4f5 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5815 punch from 0x23f1061 to 0x240059d, (0xf53c bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5816 trunc from 0x375d4f5 to 0xe30d6c0 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5817 punch from 0x4a55cfa to 0x4a611d9, (0xb4df bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5818 write 0x6c76b92 thru 0x6c869b0 (0xfe1f bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5820 read 0xa2cfd83 thru 0xa2d12ac (0x152a bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5821 read 0xa2aecda thru 0xa2b5fa6 (0x72cd bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5822 read 0x6816ccd thru 0x68229df (0xbd13 bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5823 read 0xbdd988f thru 0xbde57c8 (0xbf3a bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5824 punch from 0xdaf0a05 to 0xdaf808a, (0x7685 bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5825 write 0xe2e821b thru 0xe2ef899 (0x767f bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5826 read 0xcf9dbf8 thru 0xcfa1985 (0x3d8e bytes) 2024-09-21T23:11:50.280 INFO:teuthology.orchestra.run.smithi150.stdout:5827 trunc from 0xe30d6c0 to 0xbebe571 2024-09-21T23:11:50.283 INFO:teuthology.orchestra.run.smithi150.stdout:5828 trunc from 0xbebe571 to 0xad0e396 2024-09-21T23:11:50.292 INFO:teuthology.orchestra.run.smithi150.stdout:5829 punch from 0x5e7d7eb to 0x5e8045a, (0x2c6f bytes) 2024-09-21T23:11:50.292 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.292 INFO:teuthology.orchestra.run.smithi150.stdout:5833 write 0xa0118ac thru 0xa01fa7a (0xe1cf bytes) 2024-09-21T23:11:50.292 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:50.286+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:50.298 INFO:teuthology.orchestra.run.smithi150.stdout:5834 write 0x2a10dc2 thru 0x2a17343 (0x6582 bytes) 2024-09-21T23:11:50.303 INFO:teuthology.orchestra.run.smithi150.stdout:5835 punch from 0x28c5f5f to 0x28d4174, (0xe215 bytes) 2024-09-21T23:11:50.303 INFO:teuthology.orchestra.run.smithi150.stdout:5836 write 0x266ab70 thru 0x2676302 (0xb793 bytes) 2024-09-21T23:11:50.311 INFO:teuthology.orchestra.run.smithi150.stdout:5838 trunc from 0xad0e396 to 0x9db5a1d 2024-09-21T23:11:50.345 INFO:teuthology.orchestra.run.smithi150.stdout:5839 writesame 0xdabc250 thru 0xdabccd8 data_size 0x1d(0xa89 bytes) 2024-09-21T23:11:50.346 INFO:teuthology.orchestra.run.smithi150.stdout:5840 trunc from 0xdabccd9 to 0xe3e046b 2024-09-21T23:11:50.354 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.354 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.354 INFO:teuthology.orchestra.run.smithi150.stdout:5846 writesame 0xa7e49fe thru 0xa7e4fab data_size 0x1(0x5ae bytes) 2024-09-21T23:11:50.359 INFO:teuthology.orchestra.run.smithi150.stdout:5847 read 0x1c171ff thru 0x1c1cb62 (0x5964 bytes) 2024-09-21T23:11:50.360 INFO:teuthology.orchestra.run.smithi150.stdout:5848 write 0x3e37a94 thru 0x3e460fd (0xe66a bytes) 2024-09-21T23:11:50.365 INFO:teuthology.orchestra.run.smithi150.stdout:5849 write 0xba49ed9 thru 0xba5116e (0x7296 bytes) 2024-09-21T23:11:50.370 INFO:teuthology.orchestra.run.smithi150.stdout:5850 read 0xaf380c8 thru 0xaf422c5 (0xa1fe bytes) 2024-09-21T23:11:50.386 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:50.381+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:50.401 INFO:teuthology.orchestra.run.smithi150.stdout:5851 trunc from 0xe3e046b to 0x2906c5 2024-09-21T23:11:50.421 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:50.421 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:50.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:116331 3.18 e110/63 3:196e0317:::rbd_object_map.114969d6b98:head v 110'858, mlcod=110'855)", 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.417926+0000", 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0037457179999999999, 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037626819999999998, 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.417926+0000", 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.417926+0000", 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.417928+0000", 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.39e-06 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.417938+0000", 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0122999999999999e-05 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.417959+0000", 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1333000000000001e-05 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.417971+0000", 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1786e-05 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.418126+0000", 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015469999999999999 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.418183+0000", 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6805000000000002e-05 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4425.0:116332 3.18 e110/63 3:196e0317:::rbd_object_map.114969d6b98:head v 110'859, mlcod=110'855)", 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.419817+0000", 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.001855415, 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001909371, 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.425 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.419817+0000", 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.419817+0000", 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.419819+0000", 2024-09-21T23:11:50.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1330000000000002e-06 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.419834+0000", 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5179e-05 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.419868+0000", 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3763000000000002e-05 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.419890+0000", 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2633000000000002e-05 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.420019+0000", 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012894999999999999 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.420075+0000", 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6001999999999997e-05 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.429 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:11:50.429 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 2 2024-09-21T23:11:50.429 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:50.439 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:50.462 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.485 INFO:teuthology.orchestra.run.smithi150.stdout:5853 write 0x7150ac2 thru 0x715f7d6 (0xed15 bytes) 2024-09-21T23:11:50.487 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:50.481+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:50.511 INFO:teuthology.orchestra.run.smithi150.stdout:5854 writesame 0xccf0a54 thru 0xccfb359 data_size 0x5(0xa906 bytes) 2024-09-21T23:11:50.511 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.511 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.522 INFO:teuthology.orchestra.run.smithi150.stdout:5858 write 0xdfc230b thru 0xdfcd765 (0xb45b bytes) 2024-09-21T23:11:50.535 INFO:teuthology.orchestra.run.smithi150.stdout:5860 write 0xe16e794 thru 0xe17a71c (0xbf89 bytes) 2024-09-21T23:11:50.542 INFO:teuthology.orchestra.run.smithi150.stdout:5861 read 0x6a7312b thru 0x6a7d87b (0xa751 bytes) 2024-09-21T23:11:50.542 INFO:teuthology.orchestra.run.smithi150.stdout:5862 write 0x8135a50 thru 0x8138c2d (0x31de bytes) 2024-09-21T23:11:50.587 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:50.582+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:50.594 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:50.594 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:50.608 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:50.693 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:50.690+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:50.752 INFO:teuthology.orchestra.run.smithi150.stdout:5864 write 0x648aa0a thru 0x6499b3e (0xf135 bytes) 2024-09-21T23:11:50.760 INFO:teuthology.orchestra.run.smithi150.stdout:5866 read 0xbbb205 thru 0xbc922c (0xe028 bytes) 2024-09-21T23:11:50.760 INFO:teuthology.orchestra.run.smithi150.stdout:5867 writesame 0x21d8484 thru 0x21e110a data_size 0x5(0x8c87 bytes) 2024-09-21T23:11:50.770 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T23:11:50.777 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.172473229000005, 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:11:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:11:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:11:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.172426561999998, 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:11:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:11:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.172352496000002, 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:11:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.172289770000006, 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.172237820000007, 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.172024734000004, 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:11:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:11:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.170140708999995, 2024-09-21T23:11:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.170092659000005, 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.170044352000005, 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.169997136999996, 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:11:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.169932908000007, 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:11:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:11:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.169880997999996, 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:11:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.165278704000002, 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:11:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:11:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.165223889000004, 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:11:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.165176165000005, 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.165109275999995, 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:11:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.161563998000005, 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.848 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.161517257, 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:11:50.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:11:50.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.161470734999995, 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 43.366774556999999, 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:50.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:50.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:50.862 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:50.862 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:50.862 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:11:50.863 INFO:teuthology.orchestra.run.smithi150.stdout:5868 trunc from 0xe17a71d to 0x72f1274 2024-09-21T23:11:50.863 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:50.857+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:50.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:50.865 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:11:50.865 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:11:50.866 INFO:teuthology.orchestra.run.smithi150.stdout:5869 trunc from 0x72f1274 to 0x41d4339 2024-09-21T23:11:50.866 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.866 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5877 punch from 0x37e8e5 to 0x37fd01, (0x141c bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5878 read 0x2a3ff12 thru 0x2a45f32 (0x6021 bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5879 write 0x9ea9e1 thru 0x9f4d4e (0xa36e bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5882 read 0x209c2ee thru 0x20ac01a (0xfd2d bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5883 write 0x6745315 thru 0x6752060 (0xcd4c bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5885 writesame 0x361e116 thru 0x3623104 data_size 0x39(0x4fef bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5888 punch from 0x13371d2 to 0x134453d, (0xd36b bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5890 read 0x17c5b41 thru 0x17c975a (0x3c1a bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5891 write 0x671197b thru 0x6712b44 (0x11ca bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5892 read 0x23eb4b8 thru 0x23f3b23 (0x866c bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5894 punch from 0x1815aa0 to 0x18193c9, (0x3929 bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:5895 read 0x5d7720a thru 0x5d7a18f (0x2f86 bytes) 2024-09-21T23:11:50.867 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.868 INFO:teuthology.orchestra.run.smithi150.stdout:5897 read 0x654809c thru 0x654a282 (0x21e7 bytes) 2024-09-21T23:11:50.868 INFO:teuthology.orchestra.run.smithi150.stdout:5898 punch from 0x2070989 to 0x2072882, (0x1ef9 bytes) 2024-09-21T23:11:50.872 INFO:teuthology.orchestra.run.smithi150.stdout:5901 writesame 0x948d691 thru 0x948e5f2 data_size 0x2(0xf62 bytes) 2024-09-21T23:11:50.890 INFO:teuthology.orchestra.run.smithi150.stdout:5903 write 0xe80fca2 thru 0xe81dbce (0xdf2d bytes) 2024-09-21T23:11:50.896 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:50.896 INFO:teuthology.orchestra.run.smithi150.stdout:5905 punch from 0xddddbc2 to 0xdde7dc9, (0xa207 bytes) 2024-09-21T23:11:50.896 INFO:teuthology.orchestra.run.smithi150.stdout:5906 punch from 0x70cd286 to 0x70cfedb, (0x2c55 bytes) 2024-09-21T23:11:50.896 INFO:teuthology.orchestra.run.smithi150.stdout:5908 write 0x4df674c thru 0x4dfb6b4 (0x4f69 bytes) 2024-09-21T23:11:50.902 INFO:teuthology.orchestra.run.smithi150.stdout:5909 read 0x459ba05 thru 0x45a5251 (0x984d bytes) 2024-09-21T23:11:50.902 INFO:teuthology.orchestra.run.smithi150.stdout:5915 read 0x7b212 thru 0x7f708 (0x44f7 bytes) 2024-09-21T23:11:50.922 INFO:teuthology.orchestra.run.smithi150.stdout:5916 trunc from 0xe81dbcf to 0x160945a 2024-09-21T23:11:50.946 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:50.955 INFO:teuthology.orchestra.run.smithi150.stdout:5918 trunc from 0x160945a to 0xa134d79 2024-09-21T23:11:50.963 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:50.958+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:50.965 INFO:teuthology.orchestra.run.smithi150.stdout:5919 read 0x6f6a4a8 thru 0x6f738b8 (0x9411 bytes) 2024-09-21T23:11:50.975 INFO:teuthology.orchestra.run.smithi150.stdout:5921 trunc from 0xa134d79 to 0x4178637 2024-09-21T23:11:51.007 INFO:teuthology.orchestra.run.smithi150.stdout:5922 writesame 0xb42fa6c thru 0xb42fd8e data_size 0x1(0x323 bytes) 2024-09-21T23:11:51.007 INFO:teuthology.orchestra.run.smithi150.stdout:5923 write 0x4e4a640 thru 0x4e52e6e (0x882f bytes) 2024-09-21T23:11:51.011 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:51.011 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:51.011 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:51.012 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:51.014 INFO:teuthology.orchestra.run.smithi150.stdout:5925 write 0x32698eb thru 0x32772c8 (0xd9de bytes) 2024-09-21T23:11:51.024 INFO:teuthology.orchestra.run.smithi150.stdout:5927 read 0x7b9aca9 thru 0x7ba8fc2 (0xe31a bytes) 2024-09-21T23:11:51.034 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:51.035 INFO:teuthology.orchestra.run.smithi150.stdout:5928 writesame 0xc13ae86 thru 0xc14559d data_size 0x4(0xa718 bytes) 2024-09-21T23:11:51.050 INFO:teuthology.orchestra.run.smithi150.stdout:5929 trunc from 0xc14559e to 0x1ab8ffa 2024-09-21T23:11:51.064 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:51.061+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:51.078 INFO:teuthology.orchestra.run.smithi150.stdout:5930 read 0x11f8d83 thru 0x11fece0 (0x5f5e bytes) 2024-09-21T23:11:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:5932 writesame 0xaeba188 thru 0xaec246d data_size 0xf(0x82e6 bytes) 2024-09-21T23:11:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:5933 read 0x68080a thru 0x681185 (0x97c bytes) 2024-09-21T23:11:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:51.103 INFO:teuthology.orchestra.run.smithi150.stdout:5936 clone 52 order 25 su 131072 sc 3 2024-09-21T23:11:51.141 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:51.141 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:51.158 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:51.159 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:51.165 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:51.162+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:51.165 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:51.165 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:51.177 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:51.177 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:51.181 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:51.194 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:51.194 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:51.217 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:11:51.218 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:11:51.219 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:51.219 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:51.243 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-21T23:11:50.766650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1759573248,"num_objects":2717,"num_object_clones":714,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":1295,"num_read":164941,"num_read_kb":819698,"num_write":19849,"num_write_kb":398474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2563,"num_bytes_recovered":186754059,"num_keys_recovered":420,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16701,"ondisk_log_size":16701,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1015464,"kb_used_data":661476,"kb_used_omap":25,"kb_used_meta":353638,"kb_avail":753959256,"statfs":{"total":773094113280,"available":772054278144,"internally_reserved":0,"allocated":677351424,"data_stored":776664947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":362125359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32773304,"num_objects":326,"num_object_clones":117,"num_object_copies":652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":196,"num_read":24304,"num_read_kb":59706,"num_write":1785,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035068"},"pg_stats":[{"pgid":"3.1f","version":"110'476","reported_seq":3331,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.309604+0000","last_change":"2024-09-21T23:09:57.384622+0000","last_active":"2024-09-21T23:11:50.309604+0000","last_peered":"2024-09-21T23:11:50.309604+0000","last_clean":"2024-09-21T23:11:50.309604+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:11:50.309604+0000","last_undegraded":"2024-09-21T23:11:50.309604+0000","last_fullsized":"2024-09-21T23:11:50.309604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:45:11.319684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018862200000000001,"stat_sum":{"num_bytes":37791467,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":5261,"num_read_kb":18153,"num_write":480,"num_write_kb":6527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"110'777","reported_seq":3598,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.409635+0000","last_change":"2024-09-21T23:09:57.368121+0000","last_active":"2024-09-21T23:11:50.409635+0000","last_peered":"2024-09-21T23:11:50.409635+0000","last_clean":"2024-09-21T23:11:50.409635+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:11:50.409635+0000","last_undegraded":"2024-09-21T23:11:50.409635+0000","last_fullsized":"2024-09-21T23:11:50.409635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:42:11.698520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010293570000000001,"stat_sum":{"num_bytes":63248736,"num_objects":79,"num_object_clones":20,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":38,"num_read":5328,"num_read_kb":41041,"num_write":1073,"num_write_kb":19705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"110'368","reported_seq":2830,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.298236+0000","last_change":"2024-09-21T23:11:05.381942+0000","last_active":"2024-09-21T23:11:50.298236+0000","last_peered":"2024-09-21T23:11:50.298236+0000","last_clean":"2024-09-21T23:11:50.298236+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:11:50.298236+0000","last_undegraded":"2024-09-21T23:11:50.298236+0000","last_fullsized":"2024-09-21T23:11:50.298236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:28.632985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057415099999999996,"stat_sum":{"num_bytes":58896896,"num_objects":81,"num_object_clones":25,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":3051,"num_read_kb":13569,"num_write":344,"num_write_kb":11718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'351","reported_seq":4788,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779301+0000","last_change":"2024-09-21T23:11:07.386724+0000","last_active":"2024-09-21T23:11:49.779301+0000","last_peered":"2024-09-21T23:11:49.779301+0000","last_clean":"2024-09-21T23:11:49.779301+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:11:49.779301+0000","last_undegraded":"2024-09-21T23:11:49.779301+0000","last_fullsized":"2024-09-21T23:11:49.779301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:51.170427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090318899999999999,"stat_sum":{"num_bytes":64746275,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":27,"num_read":4671,"num_read_kb":21904,"num_write":373,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"110'492","reported_seq":3758,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.410072+0000","last_change":"2024-09-21T23:11:07.390627+0000","last_active":"2024-09-21T23:11:50.410072+0000","last_peered":"2024-09-21T23:11:50.410072+0000","last_clean":"2024-09-21T23:11:50.410072+0000","last_became_active":"2024-09-21T23:11:07.389449+0000","last_became_peered":"2024-09-21T23:11:07.389449+0000","last_unstale":"2024-09-21T23:11:50.410072+0000","last_undegraded":"2024-09-21T23:11:50.410072+0000","last_fullsized":"2024-09-21T23:11:50.410072+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:24.246245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007694,"stat_sum":{"num_bytes":44719247,"num_objects":81,"num_object_clones":22,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":4917,"num_read_kb":27542,"num_write":620,"num_write_kb":17918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1126","reported_seq":5994,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.345072+0000","last_change":"2024-09-21T23:11:06.380298+0000","last_active":"2024-09-21T23:11:50.345072+0000","last_peered":"2024-09-21T23:11:50.345072+0000","last_clean":"2024-09-21T23:11:50.345072+0000","last_became_active":"2024-09-21T23:11:06.378658+0000","last_became_peered":"2024-09-21T23:11:06.378658+0000","last_unstale":"2024-09-21T23:11:50.345072+0000","last_undegraded":"2024-09-21T23:11:50.345072+0000","last_fullsized":"2024-09-21T23:11:50.345072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:19.021730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011201469999999999,"stat_sum":{"num_bytes":50539919,"num_objects":98,"num_object_clones":25,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":46,"num_read":10639,"num_read_kb":28441,"num_write":1552,"num_write_kb":9510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":1714050,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"110'889","reported_seq":2409,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.246116+0000","last_change":"2024-09-21T23:09:57.374916+0000","last_active":"2024-09-21T23:11:50.246116+0000","last_peered":"2024-09-21T23:11:50.246116+0000","last_clean":"2024-09-21T23:11:50.246116+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:11:50.246116+0000","last_undegraded":"2024-09-21T23:11:50.246116+0000","last_fullsized":"2024-09-21T23:11:50.246116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:14:50.264934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073844600000000002,"stat_sum":{"num_bytes":42406404,"num_objects":88,"num_object_clones":21,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":43,"num_read":5691,"num_read_kb":21440,"num_write":1387,"num_write_kb":13862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"110'880","reported_seq":4732,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.529994+0000","last_change":"2024-09-21T23:10:52.811435+0000","last_active":"2024-09-21T23:11:50.529994+0000","last_peered":"2024-09-21T23:11:50.529994+0000","last_clean":"2024-09-21T23:11:50.529994+0000","last_became_active":"2024-09-21T23:10:52.807974+0000","last_became_peered":"2024-09-21T23:10:52.807974+0000","last_unstale":"2024-09-21T23:11:50.529994+0000","last_undegraded":"2024-09-21T23:11:50.529994+0000","last_fullsized":"2024-09-21T23:11:50.529994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:34:34.870787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054973999999999995,"stat_sum":{"num_bytes":55739728,"num_objects":93,"num_object_clones":25,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":39,"num_read":7306,"num_read_kb":28278,"num_write":1284,"num_write_kb":9635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3330203,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"110'508","reported_seq":4919,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.534606+0000","last_change":"2024-09-21T23:09:57.384681+0000","last_active":"2024-09-21T23:11:50.534606+0000","last_peered":"2024-09-21T23:11:50.534606+0000","last_clean":"2024-09-21T23:11:50.534606+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:11:50.534606+0000","last_undegraded":"2024-09-21T23:11:50.534606+0000","last_fullsized":"2024-09-21T23:11:50.534606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:50:36.217664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001819836,"stat_sum":{"num_bytes":72475433,"num_objects":100,"num_object_clones":28,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":52,"num_read":5364,"num_read_kb":22494,"num_write":517,"num_write_kb":12108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"110'385","reported_seq":3713,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.154451+0000","last_change":"2024-09-21T23:11:14.541037+0000","last_active":"2024-09-21T23:11:50.154451+0000","last_peered":"2024-09-21T23:11:50.154451+0000","last_clean":"2024-09-21T23:11:50.154451+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:11:50.154451+0000","last_undegraded":"2024-09-21T23:11:50.154451+0000","last_fullsized":"2024-09-21T23:11:50.154451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:50:59.692984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049884500000000004,"stat_sum":{"num_bytes":91687265,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":48,"num_read":4809,"num_read_kb":28918,"num_write":356,"num_write_kb":12516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"110'400","reported_seq":6081,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.828913+0000","last_change":"2024-09-21T23:11:14.447450+0000","last_active":"2024-09-21T23:11:49.828913+0000","last_peered":"2024-09-21T23:11:49.828913+0000","last_clean":"2024-09-21T23:11:49.828913+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:11:49.828913+0000","last_undegraded":"2024-09-21T23:11:49.828913+0000","last_fullsized":"2024-09-21T23:11:49.828913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:10.613697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313919,"stat_sum":{"num_bytes":102693227,"num_objects":84,"num_object_clones":26,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":38,"num_read":6747,"num_read_kb":35984,"num_write":370,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"109'595","reported_seq":4224,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780304+0000","last_change":"2024-09-21T23:11:14.449399+0000","last_active":"2024-09-21T23:11:49.780304+0000","last_peered":"2024-09-21T23:11:49.780304+0000","last_clean":"2024-09-21T23:11:49.780304+0000","last_became_active":"2024-09-21T23:11:14.447876+0000","last_became_peered":"2024-09-21T23:11:14.447876+0000","last_unstale":"2024-09-21T23:11:49.780304+0000","last_undegraded":"2024-09-21T23:11:49.780304+0000","last_fullsized":"2024-09-21T23:11:49.780304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:25:21.504446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110800000000001,"stat_sum":{"num_bytes":30774704,"num_objects":79,"num_object_clones":21,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":40,"num_read":5296,"num_read_kb":13791,"num_write":764,"num_write_kb":7605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4594701,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":209,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.769160+0000","last_change":"2024-09-21T23:11:11.428751+0000","last_active":"2024-09-21T23:11:49.769160+0000","last_peered":"2024-09-21T23:11:49.769160+0000","last_clean":"2024-09-21T23:11:49.769160+0000","last_became_active":"2024-09-21T23:11:11.427806+0000","last_became_peered":"2024-09-21T23:11:11.427806+0000","last_unstale":"2024-09-21T23:11:49.769160+0000","last_undegraded":"2024-09-21T23:11:49.769160+0000","last_fullsized":"2024-09-21T23:11:49.769160+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:40.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"110'471","reported_seq":2162,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.029796+0000","last_change":"2024-09-21T23:09:57.383456+0000","last_active":"2024-09-21T23:11:50.029796+0000","last_peered":"2024-09-21T23:11:50.029796+0000","last_clean":"2024-09-21T23:11:50.029796+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:11:50.029796+0000","last_undegraded":"2024-09-21T23:11:50.029796+0000","last_fullsized":"2024-09-21T23:11:50.029796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:46.448175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069300299999999998,"stat_sum":{"num_bytes":58056524,"num_objects":76,"num_object_clones":20,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":42,"num_read":2889,"num_read_kb":24836,"num_write":554,"num_write_kb":12215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779833+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:11:49.779833+0000","last_peered":"2024-09-21T23:11:49.779833+0000","last_clean":"2024-09-21T23:11:49.779833+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:11:49.779833+0000","last_undegraded":"2024-09-21T23:11:49.779833+0000","last_fullsized":"2024-09-21T23:11:49.779833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"110'669","reported_seq":5227,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.113259+0000","last_change":"2024-09-21T23:11:15.383037+0000","last_active":"2024-09-21T23:11:50.113259+0000","last_peered":"2024-09-21T23:11:50.113259+0000","last_clean":"2024-09-21T23:11:50.113259+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:11:50.113259+0000","last_undegraded":"2024-09-21T23:11:50.113259+0000","last_fullsized":"2024-09-21T23:11:50.113259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:26.378518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333359999999999,"stat_sum":{"num_bytes":87874549,"num_objects":75,"num_object_clones":19,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":7356,"num_read_kb":29593,"num_write":973,"num_write_kb":19382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780652+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:11:49.780652+0000","last_peered":"2024-09-21T23:11:49.780652+0000","last_clean":"2024-09-21T23:11:49.780652+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:11:49.780652+0000","last_undegraded":"2024-09-21T23:11:49.780652+0000","last_fullsized":"2024-09-21T23:11:49.780652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"110'520","reported_seq":3186,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.911483+0000","last_change":"2024-09-21T23:09:57.375144+0000","last_active":"2024-09-21T23:11:49.911483+0000","last_peered":"2024-09-21T23:11:49.911483+0000","last_clean":"2024-09-21T23:11:49.911483+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:11:49.911483+0000","last_undegraded":"2024-09-21T23:11:49.911483+0000","last_fullsized":"2024-09-21T23:11:49.911483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:41:34.515343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079926499999999996,"stat_sum":{"num_bytes":31920673,"num_objects":86,"num_object_clones":23,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":38,"num_read":4555,"num_read_kb":23625,"num_write":512,"num_write_kb":10662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":199,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779922+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:11:49.779922+0000","last_peered":"2024-09-21T23:11:49.779922+0000","last_clean":"2024-09-21T23:11:49.779922+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:11:49.779922+0000","last_undegraded":"2024-09-21T23:11:49.779922+0000","last_fullsized":"2024-09-21T23:11:49.779922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"110'709","reported_seq":5500,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.070420+0000","last_change":"2024-09-21T23:09:57.378889+0000","last_active":"2024-09-21T23:11:50.070420+0000","last_peered":"2024-09-21T23:11:50.070420+0000","last_clean":"2024-09-21T23:11:50.070420+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:11:50.070420+0000","last_undegraded":"2024-09-21T23:11:50.070420+0000","last_fullsized":"2024-09-21T23:11:50.070420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:52:53.392671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000364898,"stat_sum":{"num_bytes":95866163,"num_objects":95,"num_object_clones":29,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":46,"num_read":6704,"num_read_kb":41289,"num_write":1057,"num_write_kb":18307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"110'476","reported_seq":3353,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.113489+0000","last_change":"2024-09-21T23:11:07.389570+0000","last_active":"2024-09-21T23:11:50.113489+0000","last_peered":"2024-09-21T23:11:50.113489+0000","last_clean":"2024-09-21T23:11:50.113489+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:11:50.113489+0000","last_undegraded":"2024-09-21T23:11:50.113489+0000","last_fullsized":"2024-09-21T23:11:50.113489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:57.254719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060347799999999996,"stat_sum":{"num_bytes":37474991,"num_objects":96,"num_object_clones":26,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":42,"num_read":4712,"num_read_kb":24710,"num_write":515,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":202,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779336+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:11:49.779336+0000","last_peered":"2024-09-21T23:11:49.779336+0000","last_clean":"2024-09-21T23:11:49.779336+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:11:49.779336+0000","last_undegraded":"2024-09-21T23:11:49.779336+0000","last_fullsized":"2024-09-21T23:11:49.779336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"110'401","reported_seq":3577,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779755+0000","last_change":"2024-09-21T23:09:57.374883+0000","last_active":"2024-09-21T23:11:49.779755+0000","last_peered":"2024-09-21T23:11:49.779755+0000","last_clean":"2024-09-21T23:11:49.779755+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:11:49.779755+0000","last_undegraded":"2024-09-21T23:11:49.779755+0000","last_fullsized":"2024-09-21T23:11:49.779755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:49:17.978864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060209499999999997,"stat_sum":{"num_bytes":40548728,"num_objects":89,"num_object_clones":22,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":42,"num_read":5141,"num_read_kb":19239,"num_write":389,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":199,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.767374+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:11:49.767374+0000","last_peered":"2024-09-21T23:11:49.767374+0000","last_clean":"2024-09-21T23:11:49.767374+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:11:49.767374+0000","last_undegraded":"2024-09-21T23:11:49.767374+0000","last_fullsized":"2024-09-21T23:11:49.767374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"110'357","reported_seq":4689,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.771575+0000","last_change":"2024-09-21T23:11:08.384581+0000","last_active":"2024-09-21T23:11:49.771575+0000","last_peered":"2024-09-21T23:11:49.771575+0000","last_clean":"2024-09-21T23:11:49.771575+0000","last_became_active":"2024-09-21T23:11:08.383427+0000","last_became_peered":"2024-09-21T23:11:08.383427+0000","last_unstale":"2024-09-21T23:11:49.771575+0000","last_undegraded":"2024-09-21T23:11:49.771575+0000","last_fullsized":"2024-09-21T23:11:49.771575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:35:49.871069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021610209999999999,"stat_sum":{"num_bytes":45456269,"num_objects":83,"num_object_clones":23,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":41,"num_read":4595,"num_read_kb":20187,"num_write":421,"num_write_kb":9750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1250678,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779979+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:11:49.779979+0000","last_peered":"2024-09-21T23:11:49.779979+0000","last_clean":"2024-09-21T23:11:49.779979+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:11:49.779979+0000","last_undegraded":"2024-09-21T23:11:49.779979+0000","last_fullsized":"2024-09-21T23:11:49.779979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"110'490","reported_seq":3772,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.768240+0000","last_change":"2024-09-21T23:09:57.368695+0000","last_active":"2024-09-21T23:11:49.768240+0000","last_peered":"2024-09-21T23:11:49.768240+0000","last_clean":"2024-09-21T23:11:49.768240+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:11:49.768240+0000","last_undegraded":"2024-09-21T23:11:49.768240+0000","last_fullsized":"2024-09-21T23:11:49.768240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:26:36.482533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013976220000000001,"stat_sum":{"num_bytes":35277430,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":28,"num_read":4785,"num_read_kb":19101,"num_write":598,"num_write_kb":14842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779374+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:11:49.779374+0000","last_peered":"2024-09-21T23:11:49.779374+0000","last_clean":"2024-09-21T23:11:49.779374+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:11:49.779374+0000","last_undegraded":"2024-09-21T23:11:49.779374+0000","last_fullsized":"2024-09-21T23:11:49.779374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":298,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780034+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:11:49.780034+0000","last_peered":"2024-09-21T23:11:49.780034+0000","last_clean":"2024-09-21T23:11:49.780034+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:11:49.780034+0000","last_undegraded":"2024-09-21T23:11:49.780034+0000","last_fullsized":"2024-09-21T23:11:49.780034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"110'396","reported_seq":2903,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.409054+0000","last_change":"2024-09-21T23:09:57.367811+0000","last_active":"2024-09-21T23:11:50.409054+0000","last_peered":"2024-09-21T23:11:50.409054+0000","last_clean":"2024-09-21T23:11:50.409054+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:11:50.409054+0000","last_undegraded":"2024-09-21T23:11:50.409054+0000","last_fullsized":"2024-09-21T23:11:50.409054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:55.554396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094978,"stat_sum":{"num_bytes":16418895,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":3928,"num_read_kb":9424,"num_write":427,"num_write_kb":4996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"110'430","reported_seq":3472,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.979826+0000","last_change":"2024-09-21T23:11:07.388356+0000","last_active":"2024-09-21T23:11:49.979826+0000","last_peered":"2024-09-21T23:11:49.979826+0000","last_clean":"2024-09-21T23:11:49.979826+0000","last_became_active":"2024-09-21T23:11:07.385848+0000","last_became_peered":"2024-09-21T23:11:07.385848+0000","last_unstale":"2024-09-21T23:11:49.979826+0000","last_undegraded":"2024-09-21T23:11:49.979826+0000","last_fullsized":"2024-09-21T23:11:49.979826+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:00:13.600283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084957200000000004,"stat_sum":{"num_bytes":43486173,"num_objects":70,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":5580,"num_read_kb":17333,"num_write":418,"num_write_kb":5871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1899971,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"110'480","reported_seq":5247,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.303954+0000","last_change":"2024-09-21T23:11:07.391603+0000","last_active":"2024-09-21T23:11:50.303954+0000","last_peered":"2024-09-21T23:11:50.303954+0000","last_clean":"2024-09-21T23:11:50.303954+0000","last_became_active":"2024-09-21T23:11:07.390001+0000","last_became_peered":"2024-09-21T23:11:07.390001+0000","last_unstale":"2024-09-21T23:11:50.303954+0000","last_undegraded":"2024-09-21T23:11:50.303954+0000","last_fullsized":"2024-09-21T23:11:50.303954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:40:12.930177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073170699999999995,"stat_sum":{"num_bytes":46864927,"num_objects":100,"num_object_clones":27,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":43,"num_read":6260,"num_read_kb":39755,"num_write":454,"num_write_kb":11922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":6008570,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"109'314","reported_seq":3157,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.768097+0000","last_change":"2024-09-21T23:09:57.368899+0000","last_active":"2024-09-21T23:11:49.768097+0000","last_peered":"2024-09-21T23:11:49.768097+0000","last_clean":"2024-09-21T23:11:49.768097+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:11:49.768097+0000","last_undegraded":"2024-09-21T23:11:49.768097+0000","last_fullsized":"2024-09-21T23:11:49.768097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:09.641690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014237099999999999,"stat_sum":{"num_bytes":48001919,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":38,"num_read":3248,"num_read_kb":26175,"num_write":299,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"110'452","reported_seq":1833,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.832962+0000","last_change":"2024-09-21T23:09:57.379066+0000","last_active":"2024-09-21T23:11:49.832962+0000","last_peered":"2024-09-21T23:11:49.832962+0000","last_clean":"2024-09-21T23:11:49.832962+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:11:49.832962+0000","last_undegraded":"2024-09-21T23:11:49.832962+0000","last_fullsized":"2024-09-21T23:11:49.832962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:44.815121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050204199999999996,"stat_sum":{"num_bytes":28929906,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":45,"num_read":2153,"num_read_kb":17609,"num_write":449,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"110'416","reported_seq":4573,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.411651+0000","last_change":"2024-09-21T23:11:15.391226+0000","last_active":"2024-09-21T23:11:50.411651+0000","last_peered":"2024-09-21T23:11:50.411651+0000","last_clean":"2024-09-21T23:11:50.411651+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:11:50.411651+0000","last_undegraded":"2024-09-21T23:11:50.411651+0000","last_fullsized":"2024-09-21T23:11:50.411651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:27.934421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010715919999999999,"stat_sum":{"num_bytes":71583586,"num_objects":98,"num_object_clones":25,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":48,"num_read":4688,"num_read_kb":28486,"num_write":381,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"109'376","reported_seq":3555,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780818+0000","last_change":"2024-09-21T23:11:14.447644+0000","last_active":"2024-09-21T23:11:49.780818+0000","last_peered":"2024-09-21T23:11:49.780818+0000","last_clean":"2024-09-21T23:11:49.780818+0000","last_became_active":"2024-09-21T23:11:14.446401+0000","last_became_peered":"2024-09-21T23:11:14.446401+0000","last_unstale":"2024-09-21T23:11:49.780818+0000","last_undegraded":"2024-09-21T23:11:49.780818+0000","last_fullsized":"2024-09-21T23:11:49.780818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:48:54.354599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010461890000000001,"stat_sum":{"num_bytes":53825235,"num_objects":89,"num_object_clones":24,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":43,"num_read":5547,"num_read_kb":22707,"num_write":409,"num_write_kb":9467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":11225026,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"110'492","reported_seq":2116,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.189769+0000","last_change":"2024-09-21T23:11:16.386394+0000","last_active":"2024-09-21T23:11:50.189769+0000","last_peered":"2024-09-21T23:11:50.189769+0000","last_clean":"2024-09-21T23:11:50.189769+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:11:50.189769+0000","last_undegraded":"2024-09-21T23:11:50.189769+0000","last_fullsized":"2024-09-21T23:11:50.189769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:31.921007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010405119999999999,"stat_sum":{"num_bytes":29701049,"num_objects":98,"num_object_clones":27,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":48,"num_read":2302,"num_read_kb":21640,"num_write":582,"num_write_kb":10093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"110'521","reported_seq":3386,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.410217+0000","last_change":"2024-09-21T23:09:57.494767+0000","last_active":"2024-09-21T23:11:50.410217+0000","last_peered":"2024-09-21T23:11:50.410217+0000","last_clean":"2024-09-21T23:11:50.410217+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:11:50.410217+0000","last_undegraded":"2024-09-21T23:11:50.410217+0000","last_fullsized":"2024-09-21T23:11:50.410217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:33:25.359072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071189699999999996,"stat_sum":{"num_bytes":79347949,"num_objects":94,"num_object_clones":24,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":42,"num_read":5226,"num_read_kb":36857,"num_write":588,"num_write_kb":13450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"110'518","reported_seq":6140,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.027632+0000","last_change":"2024-09-21T23:11:16.386949+0000","last_active":"2024-09-21T23:11:50.027632+0000","last_peered":"2024-09-21T23:11:50.027632+0000","last_clean":"2024-09-21T23:11:50.027632+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:11:50.027632+0000","last_undegraded":"2024-09-21T23:11:50.027632+0000","last_fullsized":"2024-09-21T23:11:50.027632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:53:11.367493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010743459999999999,"stat_sum":{"num_bytes":95003974,"num_objects":87,"num_object_clones":20,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":44,"num_read":7083,"num_read_kb":42131,"num_write":595,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'504","reported_seq":6559,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.771365+0000","last_change":"2024-09-21T23:11:15.383722+0000","last_active":"2024-09-21T23:11:49.771365+0000","last_peered":"2024-09-21T23:11:49.771365+0000","last_clean":"2024-09-21T23:11:49.771365+0000","last_became_active":"2024-09-21T23:11:15.382800+0000","last_became_peered":"2024-09-21T23:11:15.382800+0000","last_unstale":"2024-09-21T23:11:49.771365+0000","last_undegraded":"2024-09-21T23:11:49.771365+0000","last_fullsized":"2024-09-21T23:11:49.771365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:18:01.453703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021982479999999999,"stat_sum":{"num_bytes":53995207,"num_objects":83,"num_object_clones":26,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":39,"num_read":7598,"num_read_kb":39239,"num_write":596,"num_write_kb":21733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"110'342","reported_seq":1553,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.252236+0000","last_change":"2024-09-21T23:11:15.385937+0000","last_active":"2024-09-21T23:11:50.252236+0000","last_peered":"2024-09-21T23:11:50.252236+0000","last_clean":"2024-09-21T23:11:50.252236+0000","last_became_active":"2024-09-21T23:11:15.384526+0000","last_became_peered":"2024-09-21T23:11:15.384526+0000","last_unstale":"2024-09-21T23:11:50.252236+0000","last_undegraded":"2024-09-21T23:11:50.252236+0000","last_fullsized":"2024-09-21T23:11:50.252236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:56.677187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":43629413,"num_objects":79,"num_object_clones":23,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":42,"num_read":1395,"num_read_kb":14107,"num_write":351,"num_write_kb":7688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":12207810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1758982861,"num_objects":2713,"num_object_clones":714,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":1295,"num_read":164825,"num_read_kb":819598,"num_write":19688,"num_write_kb":396056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2559,"num_bytes_recovered":185573323,"num_keys_recovered":420,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":665751552,"data_stored":770583404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":16581,"ondisk_log_size":16581,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":162024,"kb_used_data":112788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94209816,"statfs":{"total":96636764160,"available":96470851584,"internally_reserved":0,"allocated":115494912,"data_stored":131013660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79568,"kb_used_data":40060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94292272,"statfs":{"total":96636764160,"available":96555286528,"internally_reserved":0,"allocated":41021440,"data_stored":48618761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113208,"kb_used_data":86180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94258632,"statfs":{"total":96636764160,"available":96520839168,"internally_reserved":0,"allocated":88248320,"data_stored":107381303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743435,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":173504,"kb_used_data":132392,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94198336,"statfs":{"total":96636764160,"available":96459096064,"internally_reserved":0,"allocated":135569408,"data_stored":153316124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166832,"kb_used_data":99384,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94205008,"statfs":{"total":96636764160,"available":96465928192,"internally_reserved":0,"allocated":101769216,"data_stored":112887341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66900,"kb_used_data":31556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304940,"statfs":{"total":96636764160,"available":96568258560,"internally_reserved":0,"allocated":32313344,"data_stored":36671556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97152,"kb_used_data":52072,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94274688,"statfs":{"total":96636764160,"available":96537280512,"internally_reserved":0,"allocated":53321728,"data_stored":58977520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":156276,"kb_used_data":107044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94215564,"statfs":{"total":96636764160,"available":96476737536,"internally_reserved":0,"allocated":109613056,"data_stored":127798682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107819008,"data_stored":126692920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51961856,"data_stored":58312519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30941184,"data_stored":35995390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":100413440,"data_stored":112231246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":134377472,"data_stored":152789085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":86933504,"data_stored":106759262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":39755776,"data_stored":48043974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":113549312,"data_stored":129759008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:11:51.246 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:11:51.249 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:51.249 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:51.271 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:51.271 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:51.271 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:51.266+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:51.292 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:51.292 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:51.311 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:51.311 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:51.326 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:51.369 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:51.366+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:51.468 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:51.461+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:51.480 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:51.480 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:51.495 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:51.539 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:11:51.540 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:11:51.557 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-21T23:11:50.766650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1759573248,"num_objects":2717,"num_object_clones":714,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":1295,"num_read":164941,"num_read_kb":819698,"num_write":19849,"num_write_kb":398474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2563,"num_bytes_recovered":186754059,"num_keys_recovered":420,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16701,"ondisk_log_size":16701,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1015464,"kb_used_data":661476,"kb_used_omap":25,"kb_used_meta":353638,"kb_avail":753959256,"statfs":{"total":773094113280,"available":772054278144,"internally_reserved":0,"allocated":677351424,"data_stored":776664947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":362125359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32773304,"num_objects":326,"num_object_clones":117,"num_object_copies":652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":196,"num_read":24304,"num_read_kb":59706,"num_write":1785,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035068"},"pg_stats":[{"pgid":"3.1f","version":"110'476","reported_seq":3331,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.309604+0000","last_change":"2024-09-21T23:09:57.384622+0000","last_active":"2024-09-21T23:11:50.309604+0000","last_peered":"2024-09-21T23:11:50.309604+0000","last_clean":"2024-09-21T23:11:50.309604+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:11:50.309604+0000","last_undegraded":"2024-09-21T23:11:50.309604+0000","last_fullsized":"2024-09-21T23:11:50.309604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:45:11.319684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018862200000000001,"stat_sum":{"num_bytes":37791467,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":5261,"num_read_kb":18153,"num_write":480,"num_write_kb":6527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"110'777","reported_seq":3598,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.409635+0000","last_change":"2024-09-21T23:09:57.368121+0000","last_active":"2024-09-21T23:11:50.409635+0000","last_peered":"2024-09-21T23:11:50.409635+0000","last_clean":"2024-09-21T23:11:50.409635+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:11:50.409635+0000","last_undegraded":"2024-09-21T23:11:50.409635+0000","last_fullsized":"2024-09-21T23:11:50.409635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:42:11.698520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010293570000000001,"stat_sum":{"num_bytes":63248736,"num_objects":79,"num_object_clones":20,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":38,"num_read":5328,"num_read_kb":41041,"num_write":1073,"num_write_kb":19705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"110'368","reported_seq":2830,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.298236+0000","last_change":"2024-09-21T23:11:05.381942+0000","last_active":"2024-09-21T23:11:50.298236+0000","last_peered":"2024-09-21T23:11:50.298236+0000","last_clean":"2024-09-21T23:11:50.298236+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:11:50.298236+0000","last_undegraded":"2024-09-21T23:11:50.298236+0000","last_fullsized":"2024-09-21T23:11:50.298236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:28.632985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057415099999999996,"stat_sum":{"num_bytes":58896896,"num_objects":81,"num_object_clones":25,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":3051,"num_read_kb":13569,"num_write":344,"num_write_kb":11718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'351","reported_seq":4788,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779301+0000","last_change":"2024-09-21T23:11:07.386724+0000","last_active":"2024-09-21T23:11:49.779301+0000","last_peered":"2024-09-21T23:11:49.779301+0000","last_clean":"2024-09-21T23:11:49.779301+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:11:49.779301+0000","last_undegraded":"2024-09-21T23:11:49.779301+0000","last_fullsized":"2024-09-21T23:11:49.779301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:51.170427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090318899999999999,"stat_sum":{"num_bytes":64746275,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":27,"num_read":4671,"num_read_kb":21904,"num_write":373,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"110'492","reported_seq":3758,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.410072+0000","last_change":"2024-09-21T23:11:07.390627+0000","last_active":"2024-09-21T23:11:50.410072+0000","last_peered":"2024-09-21T23:11:50.410072+0000","last_clean":"2024-09-21T23:11:50.410072+0000","last_became_active":"2024-09-21T23:11:07.389449+0000","last_became_peered":"2024-09-21T23:11:07.389449+0000","last_unstale":"2024-09-21T23:11:50.410072+0000","last_undegraded":"2024-09-21T23:11:50.410072+0000","last_fullsized":"2024-09-21T23:11:50.410072+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:24.246245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001007694,"stat_sum":{"num_bytes":44719247,"num_objects":81,"num_object_clones":22,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":4917,"num_read_kb":27542,"num_write":620,"num_write_kb":17918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"110'1126","reported_seq":5994,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.345072+0000","last_change":"2024-09-21T23:11:06.380298+0000","last_active":"2024-09-21T23:11:50.345072+0000","last_peered":"2024-09-21T23:11:50.345072+0000","last_clean":"2024-09-21T23:11:50.345072+0000","last_became_active":"2024-09-21T23:11:06.378658+0000","last_became_peered":"2024-09-21T23:11:06.378658+0000","last_unstale":"2024-09-21T23:11:50.345072+0000","last_undegraded":"2024-09-21T23:11:50.345072+0000","last_fullsized":"2024-09-21T23:11:50.345072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:19.021730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011201469999999999,"stat_sum":{"num_bytes":50539919,"num_objects":98,"num_object_clones":25,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":46,"num_read":10639,"num_read_kb":28441,"num_write":1552,"num_write_kb":9510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":1714050,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"110'889","reported_seq":2409,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.246116+0000","last_change":"2024-09-21T23:09:57.374916+0000","last_active":"2024-09-21T23:11:50.246116+0000","last_peered":"2024-09-21T23:11:50.246116+0000","last_clean":"2024-09-21T23:11:50.246116+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:11:50.246116+0000","last_undegraded":"2024-09-21T23:11:50.246116+0000","last_fullsized":"2024-09-21T23:11:50.246116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:14:50.264934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073844600000000002,"stat_sum":{"num_bytes":42406404,"num_objects":88,"num_object_clones":21,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":43,"num_read":5691,"num_read_kb":21440,"num_write":1387,"num_write_kb":13862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"110'880","reported_seq":4732,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.529994+0000","last_change":"2024-09-21T23:10:52.811435+0000","last_active":"2024-09-21T23:11:50.529994+0000","last_peered":"2024-09-21T23:11:50.529994+0000","last_clean":"2024-09-21T23:11:50.529994+0000","last_became_active":"2024-09-21T23:10:52.807974+0000","last_became_peered":"2024-09-21T23:10:52.807974+0000","last_unstale":"2024-09-21T23:11:50.529994+0000","last_undegraded":"2024-09-21T23:11:50.529994+0000","last_fullsized":"2024-09-21T23:11:50.529994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:34:34.870787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054973999999999995,"stat_sum":{"num_bytes":55739728,"num_objects":93,"num_object_clones":25,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":39,"num_read":7306,"num_read_kb":28278,"num_write":1284,"num_write_kb":9635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3330203,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"110'508","reported_seq":4919,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.534606+0000","last_change":"2024-09-21T23:09:57.384681+0000","last_active":"2024-09-21T23:11:50.534606+0000","last_peered":"2024-09-21T23:11:50.534606+0000","last_clean":"2024-09-21T23:11:50.534606+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:11:50.534606+0000","last_undegraded":"2024-09-21T23:11:50.534606+0000","last_fullsized":"2024-09-21T23:11:50.534606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:50:36.217664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001819836,"stat_sum":{"num_bytes":72475433,"num_objects":100,"num_object_clones":28,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":52,"num_read":5364,"num_read_kb":22494,"num_write":517,"num_write_kb":12108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"110'385","reported_seq":3713,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.154451+0000","last_change":"2024-09-21T23:11:14.541037+0000","last_active":"2024-09-21T23:11:50.154451+0000","last_peered":"2024-09-21T23:11:50.154451+0000","last_clean":"2024-09-21T23:11:50.154451+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:11:50.154451+0000","last_undegraded":"2024-09-21T23:11:50.154451+0000","last_fullsized":"2024-09-21T23:11:50.154451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:50:59.692984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049884500000000004,"stat_sum":{"num_bytes":91687265,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":48,"num_read":4809,"num_read_kb":28918,"num_write":356,"num_write_kb":12516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"110'400","reported_seq":6081,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.828913+0000","last_change":"2024-09-21T23:11:14.447450+0000","last_active":"2024-09-21T23:11:49.828913+0000","last_peered":"2024-09-21T23:11:49.828913+0000","last_clean":"2024-09-21T23:11:49.828913+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:11:49.828913+0000","last_undegraded":"2024-09-21T23:11:49.828913+0000","last_fullsized":"2024-09-21T23:11:49.828913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:10.613697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313919,"stat_sum":{"num_bytes":102693227,"num_objects":84,"num_object_clones":26,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":38,"num_read":6747,"num_read_kb":35984,"num_write":370,"num_write_kb":14897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"109'595","reported_seq":4224,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780304+0000","last_change":"2024-09-21T23:11:14.449399+0000","last_active":"2024-09-21T23:11:49.780304+0000","last_peered":"2024-09-21T23:11:49.780304+0000","last_clean":"2024-09-21T23:11:49.780304+0000","last_became_active":"2024-09-21T23:11:14.447876+0000","last_became_peered":"2024-09-21T23:11:14.447876+0000","last_unstale":"2024-09-21T23:11:49.780304+0000","last_undegraded":"2024-09-21T23:11:49.780304+0000","last_fullsized":"2024-09-21T23:11:49.780304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:25:21.504446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110800000000001,"stat_sum":{"num_bytes":30774704,"num_objects":79,"num_object_clones":21,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":40,"num_read":5296,"num_read_kb":13791,"num_write":764,"num_write_kb":7605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4594701,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":209,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.769160+0000","last_change":"2024-09-21T23:11:11.428751+0000","last_active":"2024-09-21T23:11:49.769160+0000","last_peered":"2024-09-21T23:11:49.769160+0000","last_clean":"2024-09-21T23:11:49.769160+0000","last_became_active":"2024-09-21T23:11:11.427806+0000","last_became_peered":"2024-09-21T23:11:11.427806+0000","last_unstale":"2024-09-21T23:11:49.769160+0000","last_undegraded":"2024-09-21T23:11:49.769160+0000","last_fullsized":"2024-09-21T23:11:49.769160+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:40.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"110'471","reported_seq":2162,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.029796+0000","last_change":"2024-09-21T23:09:57.383456+0000","last_active":"2024-09-21T23:11:50.029796+0000","last_peered":"2024-09-21T23:11:50.029796+0000","last_clean":"2024-09-21T23:11:50.029796+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:11:50.029796+0000","last_undegraded":"2024-09-21T23:11:50.029796+0000","last_fullsized":"2024-09-21T23:11:50.029796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:46.448175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069300299999999998,"stat_sum":{"num_bytes":58056524,"num_objects":76,"num_object_clones":20,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":42,"num_read":2889,"num_read_kb":24836,"num_write":554,"num_write_kb":12215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":198,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779833+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:11:49.779833+0000","last_peered":"2024-09-21T23:11:49.779833+0000","last_clean":"2024-09-21T23:11:49.779833+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:11:49.779833+0000","last_undegraded":"2024-09-21T23:11:49.779833+0000","last_fullsized":"2024-09-21T23:11:49.779833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"110'669","reported_seq":5227,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.113259+0000","last_change":"2024-09-21T23:11:15.383037+0000","last_active":"2024-09-21T23:11:50.113259+0000","last_peered":"2024-09-21T23:11:50.113259+0000","last_clean":"2024-09-21T23:11:50.113259+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:11:50.113259+0000","last_undegraded":"2024-09-21T23:11:50.113259+0000","last_fullsized":"2024-09-21T23:11:50.113259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:26.378518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010333359999999999,"stat_sum":{"num_bytes":87874549,"num_objects":75,"num_object_clones":19,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":7356,"num_read_kb":29593,"num_write":973,"num_write_kb":19382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780652+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:11:49.780652+0000","last_peered":"2024-09-21T23:11:49.780652+0000","last_clean":"2024-09-21T23:11:49.780652+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:11:49.780652+0000","last_undegraded":"2024-09-21T23:11:49.780652+0000","last_fullsized":"2024-09-21T23:11:49.780652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"110'520","reported_seq":3186,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.911483+0000","last_change":"2024-09-21T23:09:57.375144+0000","last_active":"2024-09-21T23:11:49.911483+0000","last_peered":"2024-09-21T23:11:49.911483+0000","last_clean":"2024-09-21T23:11:49.911483+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:11:49.911483+0000","last_undegraded":"2024-09-21T23:11:49.911483+0000","last_fullsized":"2024-09-21T23:11:49.911483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:41:34.515343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079926499999999996,"stat_sum":{"num_bytes":31920673,"num_objects":86,"num_object_clones":23,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":38,"num_read":4555,"num_read_kb":23625,"num_write":512,"num_write_kb":10662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":199,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779922+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:11:49.779922+0000","last_peered":"2024-09-21T23:11:49.779922+0000","last_clean":"2024-09-21T23:11:49.779922+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:11:49.779922+0000","last_undegraded":"2024-09-21T23:11:49.779922+0000","last_fullsized":"2024-09-21T23:11:49.779922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"110'709","reported_seq":5500,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.070420+0000","last_change":"2024-09-21T23:09:57.378889+0000","last_active":"2024-09-21T23:11:50.070420+0000","last_peered":"2024-09-21T23:11:50.070420+0000","last_clean":"2024-09-21T23:11:50.070420+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:11:50.070420+0000","last_undegraded":"2024-09-21T23:11:50.070420+0000","last_fullsized":"2024-09-21T23:11:50.070420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:52:53.392671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000364898,"stat_sum":{"num_bytes":95866163,"num_objects":95,"num_object_clones":29,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":46,"num_read":6704,"num_read_kb":41289,"num_write":1057,"num_write_kb":18307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"110'476","reported_seq":3353,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.113489+0000","last_change":"2024-09-21T23:11:07.389570+0000","last_active":"2024-09-21T23:11:50.113489+0000","last_peered":"2024-09-21T23:11:50.113489+0000","last_clean":"2024-09-21T23:11:50.113489+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:11:50.113489+0000","last_undegraded":"2024-09-21T23:11:50.113489+0000","last_fullsized":"2024-09-21T23:11:50.113489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:57.254719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060347799999999996,"stat_sum":{"num_bytes":37474991,"num_objects":96,"num_object_clones":26,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":42,"num_read":4712,"num_read_kb":24710,"num_write":515,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":202,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779336+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:11:49.779336+0000","last_peered":"2024-09-21T23:11:49.779336+0000","last_clean":"2024-09-21T23:11:49.779336+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:11:49.779336+0000","last_undegraded":"2024-09-21T23:11:49.779336+0000","last_fullsized":"2024-09-21T23:11:49.779336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"110'401","reported_seq":3577,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779755+0000","last_change":"2024-09-21T23:09:57.374883+0000","last_active":"2024-09-21T23:11:49.779755+0000","last_peered":"2024-09-21T23:11:49.779755+0000","last_clean":"2024-09-21T23:11:49.779755+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:11:49.779755+0000","last_undegraded":"2024-09-21T23:11:49.779755+0000","last_fullsized":"2024-09-21T23:11:49.779755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:49:17.978864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060209499999999997,"stat_sum":{"num_bytes":40548728,"num_objects":89,"num_object_clones":22,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":42,"num_read":5141,"num_read_kb":19239,"num_write":389,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":199,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.767374+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:11:49.767374+0000","last_peered":"2024-09-21T23:11:49.767374+0000","last_clean":"2024-09-21T23:11:49.767374+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:11:49.767374+0000","last_undegraded":"2024-09-21T23:11:49.767374+0000","last_fullsized":"2024-09-21T23:11:49.767374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"110'357","reported_seq":4689,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.771575+0000","last_change":"2024-09-21T23:11:08.384581+0000","last_active":"2024-09-21T23:11:49.771575+0000","last_peered":"2024-09-21T23:11:49.771575+0000","last_clean":"2024-09-21T23:11:49.771575+0000","last_became_active":"2024-09-21T23:11:08.383427+0000","last_became_peered":"2024-09-21T23:11:08.383427+0000","last_unstale":"2024-09-21T23:11:49.771575+0000","last_undegraded":"2024-09-21T23:11:49.771575+0000","last_fullsized":"2024-09-21T23:11:49.771575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:35:49.871069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021610209999999999,"stat_sum":{"num_bytes":45456269,"num_objects":83,"num_object_clones":23,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":41,"num_read":4595,"num_read_kb":20187,"num_write":421,"num_write_kb":9750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1250678,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779979+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:11:49.779979+0000","last_peered":"2024-09-21T23:11:49.779979+0000","last_clean":"2024-09-21T23:11:49.779979+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:11:49.779979+0000","last_undegraded":"2024-09-21T23:11:49.779979+0000","last_fullsized":"2024-09-21T23:11:49.779979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"110'490","reported_seq":3772,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.768240+0000","last_change":"2024-09-21T23:09:57.368695+0000","last_active":"2024-09-21T23:11:49.768240+0000","last_peered":"2024-09-21T23:11:49.768240+0000","last_clean":"2024-09-21T23:11:49.768240+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:11:49.768240+0000","last_undegraded":"2024-09-21T23:11:49.768240+0000","last_fullsized":"2024-09-21T23:11:49.768240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:26:36.482533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013976220000000001,"stat_sum":{"num_bytes":35277430,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":28,"num_read":4785,"num_read_kb":19101,"num_write":598,"num_write_kb":14842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.779374+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:11:49.779374+0000","last_peered":"2024-09-21T23:11:49.779374+0000","last_clean":"2024-09-21T23:11:49.779374+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:11:49.779374+0000","last_undegraded":"2024-09-21T23:11:49.779374+0000","last_fullsized":"2024-09-21T23:11:49.779374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":298,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780034+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:11:49.780034+0000","last_peered":"2024-09-21T23:11:49.780034+0000","last_clean":"2024-09-21T23:11:49.780034+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:11:49.780034+0000","last_undegraded":"2024-09-21T23:11:49.780034+0000","last_fullsized":"2024-09-21T23:11:49.780034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"110'396","reported_seq":2903,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.409054+0000","last_change":"2024-09-21T23:09:57.367811+0000","last_active":"2024-09-21T23:11:50.409054+0000","last_peered":"2024-09-21T23:11:50.409054+0000","last_clean":"2024-09-21T23:11:50.409054+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:11:50.409054+0000","last_undegraded":"2024-09-21T23:11:50.409054+0000","last_fullsized":"2024-09-21T23:11:50.409054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:55.554396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094978,"stat_sum":{"num_bytes":16418895,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":3928,"num_read_kb":9424,"num_write":427,"num_write_kb":4996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"110'430","reported_seq":3472,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.979826+0000","last_change":"2024-09-21T23:11:07.388356+0000","last_active":"2024-09-21T23:11:49.979826+0000","last_peered":"2024-09-21T23:11:49.979826+0000","last_clean":"2024-09-21T23:11:49.979826+0000","last_became_active":"2024-09-21T23:11:07.385848+0000","last_became_peered":"2024-09-21T23:11:07.385848+0000","last_unstale":"2024-09-21T23:11:49.979826+0000","last_undegraded":"2024-09-21T23:11:49.979826+0000","last_fullsized":"2024-09-21T23:11:49.979826+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:00:13.600283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084957200000000004,"stat_sum":{"num_bytes":43486173,"num_objects":70,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":5580,"num_read_kb":17333,"num_write":418,"num_write_kb":5871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1899971,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"110'480","reported_seq":5247,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.303954+0000","last_change":"2024-09-21T23:11:07.391603+0000","last_active":"2024-09-21T23:11:50.303954+0000","last_peered":"2024-09-21T23:11:50.303954+0000","last_clean":"2024-09-21T23:11:50.303954+0000","last_became_active":"2024-09-21T23:11:07.390001+0000","last_became_peered":"2024-09-21T23:11:07.390001+0000","last_unstale":"2024-09-21T23:11:50.303954+0000","last_undegraded":"2024-09-21T23:11:50.303954+0000","last_fullsized":"2024-09-21T23:11:50.303954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:40:12.930177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073170699999999995,"stat_sum":{"num_bytes":46864927,"num_objects":100,"num_object_clones":27,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":43,"num_read":6260,"num_read_kb":39755,"num_write":454,"num_write_kb":11922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":6008570,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"109'314","reported_seq":3157,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.768097+0000","last_change":"2024-09-21T23:09:57.368899+0000","last_active":"2024-09-21T23:11:49.768097+0000","last_peered":"2024-09-21T23:11:49.768097+0000","last_clean":"2024-09-21T23:11:49.768097+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:11:49.768097+0000","last_undegraded":"2024-09-21T23:11:49.768097+0000","last_fullsized":"2024-09-21T23:11:49.768097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:09.641690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014237099999999999,"stat_sum":{"num_bytes":48001919,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":38,"num_read":3248,"num_read_kb":26175,"num_write":299,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"110'452","reported_seq":1833,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.832962+0000","last_change":"2024-09-21T23:09:57.379066+0000","last_active":"2024-09-21T23:11:49.832962+0000","last_peered":"2024-09-21T23:11:49.832962+0000","last_clean":"2024-09-21T23:11:49.832962+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:11:49.832962+0000","last_undegraded":"2024-09-21T23:11:49.832962+0000","last_fullsized":"2024-09-21T23:11:49.832962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:44.815121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050204199999999996,"stat_sum":{"num_bytes":28929906,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":45,"num_read":2153,"num_read_kb":17609,"num_write":449,"num_write_kb":9141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"110'416","reported_seq":4573,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.411651+0000","last_change":"2024-09-21T23:11:15.391226+0000","last_active":"2024-09-21T23:11:50.411651+0000","last_peered":"2024-09-21T23:11:50.411651+0000","last_clean":"2024-09-21T23:11:50.411651+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:11:50.411651+0000","last_undegraded":"2024-09-21T23:11:50.411651+0000","last_fullsized":"2024-09-21T23:11:50.411651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:27.934421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010715919999999999,"stat_sum":{"num_bytes":71583586,"num_objects":98,"num_object_clones":25,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":48,"num_read":4688,"num_read_kb":28486,"num_write":381,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"109'376","reported_seq":3555,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.780818+0000","last_change":"2024-09-21T23:11:14.447644+0000","last_active":"2024-09-21T23:11:49.780818+0000","last_peered":"2024-09-21T23:11:49.780818+0000","last_clean":"2024-09-21T23:11:49.780818+0000","last_became_active":"2024-09-21T23:11:14.446401+0000","last_became_peered":"2024-09-21T23:11:14.446401+0000","last_unstale":"2024-09-21T23:11:49.780818+0000","last_undegraded":"2024-09-21T23:11:49.780818+0000","last_fullsized":"2024-09-21T23:11:49.780818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:48:54.354599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010461890000000001,"stat_sum":{"num_bytes":53825235,"num_objects":89,"num_object_clones":24,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":43,"num_read":5547,"num_read_kb":22707,"num_write":409,"num_write_kb":9467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":11225026,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"110'492","reported_seq":2116,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.189769+0000","last_change":"2024-09-21T23:11:16.386394+0000","last_active":"2024-09-21T23:11:50.189769+0000","last_peered":"2024-09-21T23:11:50.189769+0000","last_clean":"2024-09-21T23:11:50.189769+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:11:50.189769+0000","last_undegraded":"2024-09-21T23:11:50.189769+0000","last_fullsized":"2024-09-21T23:11:50.189769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:31.921007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010405119999999999,"stat_sum":{"num_bytes":29701049,"num_objects":98,"num_object_clones":27,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":48,"num_read":2302,"num_read_kb":21640,"num_write":582,"num_write_kb":10093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"110'521","reported_seq":3386,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.410217+0000","last_change":"2024-09-21T23:09:57.494767+0000","last_active":"2024-09-21T23:11:50.410217+0000","last_peered":"2024-09-21T23:11:50.410217+0000","last_clean":"2024-09-21T23:11:50.410217+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:11:50.410217+0000","last_undegraded":"2024-09-21T23:11:50.410217+0000","last_fullsized":"2024-09-21T23:11:50.410217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:33:25.359072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071189699999999996,"stat_sum":{"num_bytes":79347949,"num_objects":94,"num_object_clones":24,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":42,"num_read":5226,"num_read_kb":36857,"num_write":588,"num_write_kb":13450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"110'518","reported_seq":6140,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.027632+0000","last_change":"2024-09-21T23:11:16.386949+0000","last_active":"2024-09-21T23:11:50.027632+0000","last_peered":"2024-09-21T23:11:50.027632+0000","last_clean":"2024-09-21T23:11:50.027632+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:11:50.027632+0000","last_undegraded":"2024-09-21T23:11:50.027632+0000","last_fullsized":"2024-09-21T23:11:50.027632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:53:11.367493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010743459999999999,"stat_sum":{"num_bytes":95003974,"num_objects":87,"num_object_clones":20,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":44,"num_read":7083,"num_read_kb":42131,"num_write":595,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'504","reported_seq":6559,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:49.771365+0000","last_change":"2024-09-21T23:11:15.383722+0000","last_active":"2024-09-21T23:11:49.771365+0000","last_peered":"2024-09-21T23:11:49.771365+0000","last_clean":"2024-09-21T23:11:49.771365+0000","last_became_active":"2024-09-21T23:11:15.382800+0000","last_became_peered":"2024-09-21T23:11:15.382800+0000","last_unstale":"2024-09-21T23:11:49.771365+0000","last_undegraded":"2024-09-21T23:11:49.771365+0000","last_fullsized":"2024-09-21T23:11:49.771365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:18:01.453703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021982479999999999,"stat_sum":{"num_bytes":53995207,"num_objects":83,"num_object_clones":26,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":39,"num_read":7598,"num_read_kb":39239,"num_write":596,"num_write_kb":21733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"110'342","reported_seq":1553,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-21T23:11:50.252236+0000","last_change":"2024-09-21T23:11:15.385937+0000","last_active":"2024-09-21T23:11:50.252236+0000","last_peered":"2024-09-21T23:11:50.252236+0000","last_clean":"2024-09-21T23:11:50.252236+0000","last_became_active":"2024-09-21T23:11:15.384526+0000","last_became_peered":"2024-09-21T23:11:15.384526+0000","last_unstale":"2024-09-21T23:11:50.252236+0000","last_undegraded":"2024-09-21T23:11:50.252236+0000","last_fullsized":"2024-09-21T23:11:50.252236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:56.677187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":43629413,"num_objects":79,"num_object_clones":23,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":42,"num_read":1395,"num_read_kb":14107,"num_write":351,"num_write_kb":7688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":12207810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1758982861,"num_objects":2713,"num_object_clones":714,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":1295,"num_read":164825,"num_read_kb":819598,"num_write":19688,"num_write_kb":396056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2559,"num_bytes_recovered":185573323,"num_keys_recovered":420,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":665751552,"data_stored":770583404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":16581,"ondisk_log_size":16581,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":162024,"kb_used_data":112788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94209816,"statfs":{"total":96636764160,"available":96470851584,"internally_reserved":0,"allocated":115494912,"data_stored":131013660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79568,"kb_used_data":40060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94292272,"statfs":{"total":96636764160,"available":96555286528,"internally_reserved":0,"allocated":41021440,"data_stored":48618761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113208,"kb_used_data":86180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94258632,"statfs":{"total":96636764160,"available":96520839168,"internally_reserved":0,"allocated":88248320,"data_stored":107381303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743435,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":173504,"kb_used_data":132392,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94198336,"statfs":{"total":96636764160,"available":96459096064,"internally_reserved":0,"allocated":135569408,"data_stored":153316124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166832,"kb_used_data":99384,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94205008,"statfs":{"total":96636764160,"available":96465928192,"internally_reserved":0,"allocated":101769216,"data_stored":112887341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66900,"kb_used_data":31556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304940,"statfs":{"total":96636764160,"available":96568258560,"internally_reserved":0,"allocated":32313344,"data_stored":36671556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97152,"kb_used_data":52072,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94274688,"statfs":{"total":96636764160,"available":96537280512,"internally_reserved":0,"allocated":53321728,"data_stored":58977520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":156276,"kb_used_data":107044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94215564,"statfs":{"total":96636764160,"available":96476737536,"internally_reserved":0,"allocated":109613056,"data_stored":127798682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107819008,"data_stored":126692920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51961856,"data_stored":58312519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30941184,"data_stored":35995390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":100413440,"data_stored":112231246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":134377472,"data_stored":152789085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":86933504,"data_stored":106759262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":39755776,"data_stored":48043974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":113549312,"data_stored":129759008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:11:51.559 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:11:51.559 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:51.559 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T23:11:51.559 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:11:51.559 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:11:51.559 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T23:11:51.605 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:51.602+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:51.618 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:51.618 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:51.631 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:51.670 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:51.665+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:51.725 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:11:51.725 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:51.771 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:51.769+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:51.792 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:11:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:51.814 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:51.871 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:51.866+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:51.930 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:51.930 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:51.944 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:51.972 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:51.965+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:52.073 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:52.070+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:52.097 INFO:teuthology.orchestra.run.smithi150.stdout:truncating image image_client.0-clone51 from 0xaec246e (overlap 0x2906c5) to 0xc4ed4 2024-09-21T23:11:52.107 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:52.107 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:52.122 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:52.213 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:52.210+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:52.274 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:52.270+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:52.289 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:52.289 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:52.303 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:52.375 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:52.369+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:52.400 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-09-21T23:11:52.403 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:52.403 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:52.403 INFO:teuthology.orchestra.run.smithi150.stdout:5939 read 0x84abd77 thru 0x84b51c3 (0x944d bytes) 2024-09-21T23:11:52.404 INFO:teuthology.orchestra.run.smithi150.stdout:5940 writesame 0x43dab9f thru 0x43e6a06 data_size 0x1(0xbe68 bytes) 2024-09-21T23:11:52.451 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:52.451 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:52.463 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:52.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:52.479 INFO:teuthology.orchestra.run.smithi150.stdout:5942 trunc from 0xaec246e to 0x40f5577 2024-09-21T23:11:52.505 INFO:teuthology.orchestra.run.smithi150.stdout:5943 read 0x2d5cadf thru 0x2d6837e (0xb8a0 bytes) 2024-09-21T23:11:52.509 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:52.505+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:52.541 INFO:teuthology.orchestra.run.smithi150.stdout:5945 write 0xda05a1b thru 0xda084e8 (0x2ace bytes) 2024-09-21T23:11:52.546 INFO:teuthology.orchestra.run.smithi150.stdout:5946 read 0x703eb86 thru 0x703f518 (0x993 bytes) 2024-09-21T23:11:52.546 INFO:teuthology.orchestra.run.smithi150.stdout:5947 read 0xa397d6 thru 0xa40ead (0x76d8 bytes) 2024-09-21T23:11:52.547 INFO:teuthology.orchestra.run.smithi150.stdout:5948 punch from 0x6ff64d3 to 0x6ffb2eb, (0x4e18 bytes) 2024-09-21T23:11:52.548 INFO:teuthology.orchestra.run.smithi150.stdout:5951 read 0x49da82e thru 0x49e81d6 (0xd9a9 bytes) 2024-09-21T23:11:52.548 INFO:teuthology.orchestra.run.smithi150.stdout:5953 read 0x42620da thru 0x426c8d8 (0xa7ff bytes) 2024-09-21T23:11:52.549 INFO:teuthology.orchestra.run.smithi150.stdout:5956 write 0xbb3fc68 thru 0xbb43f45 (0x42de bytes) 2024-09-21T23:11:52.557 INFO:teuthology.orchestra.run.smithi150.stdout:5957 read 0x5f12eed thru 0x5f20f33 (0xe047 bytes) 2024-09-21T23:11:52.558 INFO:teuthology.orchestra.run.smithi150.stdout:5959 writesame 0x6f6c214 thru 0x6f77c27 data_size 0x2(0xba14 bytes) 2024-09-21T23:11:52.574 INFO:teuthology.orchestra.run.smithi150.stdout:5962 write 0x7bd29cf thru 0x7be0224 (0xd856 bytes) 2024-09-21T23:11:52.576 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:52.573+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:52.579 INFO:teuthology.orchestra.run.smithi150.stdout:5963 writesame 0xb9e7242 thru 0xb9f6b10 data_size 0x1(0xf8cf bytes) 2024-09-21T23:11:52.633 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:52.633 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:52.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:52.660 INFO:teuthology.orchestra.run.smithi150.stdout:5964 read 0xbf6c3c5 thru 0xbf76aa2 (0xa6de bytes) 2024-09-21T23:11:52.683 INFO:teuthology.orchestra.run.smithi150.stdout:5965 trunc from 0xda084e9 to 0x57e19c1 2024-09-21T23:11:52.689 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:52.685+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:52.709 INFO:teuthology.orchestra.run.smithi150.stdout:5969 trunc from 0x57e19c1 to 0x487e1e3 2024-09-21T23:11:52.722 INFO:teuthology.orchestra.run.smithi150.stdout:5970 punch from 0x197f18e to 0x1986b9c, (0x7a0e bytes) 2024-09-21T23:11:52.756 INFO:teuthology.orchestra.run.smithi150.stdout:5971 write 0xbf82263 thru 0xbf83e30 (0x1bce bytes) 2024-09-21T23:11:52.762 INFO:teuthology.orchestra.run.smithi150.stdout:5972 read 0xa125687 thru 0xa12d479 (0x7df3 bytes) 2024-09-21T23:11:52.763 INFO:teuthology.orchestra.run.smithi150.stdout:5973 punch from 0xb40b8ae to 0xb4179d7, (0xc129 bytes) 2024-09-21T23:11:52.763 INFO:teuthology.orchestra.run.smithi150.stdout:5975 punch from 0x55774dd to 0x558077a, (0x929d bytes) 2024-09-21T23:11:52.763 INFO:teuthology.orchestra.run.smithi150.stdout:5976 write 0x72a9b1 thru 0x7399fa (0xf04a bytes) 2024-09-21T23:11:52.767 INFO:teuthology.orchestra.run.smithi150.stdout:5977 write 0xc6739 thru 0xc9588 (0x2e50 bytes) 2024-09-21T23:11:52.771 INFO:teuthology.orchestra.run.smithi150.stdout:5979 write 0x872488c thru 0x872cd4e (0x84c3 bytes) 2024-09-21T23:11:52.779 INFO:teuthology.orchestra.run.smithi150.stdout:5980 punch from 0x26519f0 to 0x2653d48, (0x2358 bytes) 2024-09-21T23:11:52.779 INFO:teuthology.orchestra.run.smithi150.stdout:5981 punch from 0x7e111bc to 0x7e19c9f, (0x8ae3 bytes) 2024-09-21T23:11:52.779 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:52.774+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:52.779 INFO:teuthology.orchestra.run.smithi150.stdout:5982 read 0x8263765 thru 0x82638d2 (0x16e bytes) 2024-09-21T23:11:52.780 INFO:teuthology.orchestra.run.smithi150.stdout:5983 read 0xa04a027 thru 0xa059b1c (0xfaf6 bytes) 2024-09-21T23:11:52.780 INFO:teuthology.orchestra.run.smithi150.stdout:5984 punch from 0x1379104 to 0x13815d9, (0x84d5 bytes) 2024-09-21T23:11:52.781 INFO:teuthology.orchestra.run.smithi150.stdout:5985 writesame 0xc039a4 thru 0xc0532e data_size 0x1(0x198b bytes) 2024-09-21T23:11:52.791 INFO:teuthology.orchestra.run.smithi150.stdout:5986 read 0xb7ef8c8 thru 0xb7f3b47 (0x4280 bytes) 2024-09-21T23:11:52.792 INFO:teuthology.orchestra.run.smithi150.stdout:5987 read 0x263296e thru 0x2640468 (0xdafb bytes) 2024-09-21T23:11:52.793 INFO:teuthology.orchestra.run.smithi150.stdout:5988 write 0x299ba95 thru 0x29a800e (0xc57a bytes) 2024-09-21T23:11:52.812 INFO:teuthology.orchestra.run.smithi150.stdout:5989 write 0xcf6af9a thru 0xcf7928b (0xe2f2 bytes) 2024-09-21T23:11:52.815 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:52.815 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:52.819 INFO:teuthology.orchestra.run.smithi150.stdout:5991 read 0x9eabd57 thru 0x9eac4a4 (0x74e bytes) 2024-09-21T23:11:52.819 INFO:teuthology.orchestra.run.smithi150.stdout:5992 writesame 0x31a9953 thru 0x31b3e3d data_size 0x3(0xa4eb bytes) 2024-09-21T23:11:52.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:52.832 INFO:teuthology.orchestra.run.smithi150.stdout:5993 write 0x1a77606 thru 0x1a7f3b6 (0x7db1 bytes) 2024-09-21T23:11:52.853 INFO:teuthology.orchestra.run.smithi150.stdout:5994 writesame 0xeb594a2 thru 0xeb67308 data_size 0x3b(0xde67 bytes) 2024-09-21T23:11:52.853 INFO:teuthology.orchestra.run.smithi150.stdout:5995 write 0xa2e0df4 thru 0xa2e299b (0x1ba8 bytes) 2024-09-21T23:11:52.855 INFO:teuthology.orchestra.run.smithi150.stdout:5996 write 0x8ec2211 thru 0x8ec74f6 (0x52e6 bytes) 2024-09-21T23:11:52.858 INFO:teuthology.orchestra.run.smithi150.stdout:compare and write disabled 2024-09-21T23:11:52.858 INFO:teuthology.orchestra.run.smithi150.stdout:5998 read 0xd3bc409 thru 0xd3c2112 (0x5d0a bytes) 2024-09-21T23:11:52.858 INFO:teuthology.orchestra.run.smithi150.stdout:5999 writesame 0xb9ed3de thru 0xb9fa9fb data_size 0x1(0xd61e bytes) 2024-09-21T23:11:52.880 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:52.873+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:52.924 INFO:teuthology.orchestra.run.smithi150.stdout:6000 read 0x1059f26 thru 0x106949b (0xf576 bytes) 2024-09-21T23:11:52.951 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:52.952 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:52.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:52.981 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:52.978+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:53.082 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:53.077+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:53.134 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:53.134 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:53.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:53.197 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:53.193+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:53.271 INFO:teuthology.orchestra.run.smithi150.stdout:checking clone #51, image image_client.0-clone51 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent52 2024-09-21T23:11:53.283 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:53.278+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:53.311 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:53.324 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:53.324 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:53.339 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:53.384 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:53.378+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:53.465 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:53.465 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:53.477 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:53.477 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:53.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:53.485 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:53.482+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:53.495 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:53.495 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:53.512 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:53.512 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:53.529 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:53.529 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:53.552 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:53.552 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:53.574 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:53.574 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:53.586 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:53.582+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:53.594 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:53.594 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:53.605 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:53.605 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:53.615 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:53.615 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:53.622 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:53.686 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:53.682+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:53.753 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:53.753 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:53.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:53.753 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.566416992, 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.754 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.566313906, 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.758 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.566246557, 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.762 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.49105962100001, 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.766 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.770 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.491003853, 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:11:53.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.774 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.490972314, 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:11:53.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.49093956500001, 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.779 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:53.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:53.783 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.49091267199999, 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.787 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.490887399, 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.788 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.789 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.490856272, 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.792 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:53.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:11:53.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.490212916, 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:53.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.800 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.485026989, 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.801 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:11:53.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.48500550599999, 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:11:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:11:53.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:11:53.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.48498121999999, 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.810 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:11:53.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.484963553, 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.814 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.818 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.48493996800001, 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:53.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.822 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.48492395300001, 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:11:53.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.484910122, 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.827 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:53.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:11:53.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:11:53.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.484878761, 2024-09-21T23:11:53.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:11:53.835 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.48481033900001, 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:53.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:11:53.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.840 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:53.840 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:53.840 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:53.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:11:53.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:53.842 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:53.838+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:53.942 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:53.938+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:53.971 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:53.971 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:53.971 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:53.971 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:53.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:54.043 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:54.038+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:54.103 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:54.103 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:54.103 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:54.103 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:54.104 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:54.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:54.164 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:54.161+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:54.244 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:54.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.851102005000001, 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:11:54.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:11:54.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:54.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71501 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7200768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381156939999997, 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069501725, 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71501 2024-09-21T23:11:54.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860150+0000", 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-07 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860152+0000", 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9870000000000002e-06 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860158+0000", 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T23:11:54.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860162+0000", 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9180000000000002e-06 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929487+0000", 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069324330000000003 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929580+0000", 2024-09-21T23:11:54.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2839999999999999e-05 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929651+0000", 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1416999999999997e-05 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381137136, 2024-09-21T23:11:54.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:54.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:11:54.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:11:54.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381113980999999, 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.259 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:11:54.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:11:54.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:11:54.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381091763999997, 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.263 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:11:54.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:11:54.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.267 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381071153000001, 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381051528999997, 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.272 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381030592000002, 2024-09-21T23:11:54.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:11:54.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:11:54.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.381008629, 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.281 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:11:54.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:11:54.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380989204999999, 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:11:54.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:11:54.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380970064000003, 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.289 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:54.293 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380943614000003, 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:54.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:11:54.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:11:54.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380924094000001, 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380901287999997, 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:11:54.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:11:54.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380880816000001, 2024-09-21T23:11:54.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.310 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380805051000003, 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.311 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:11:54.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.380781378999998, 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.315 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.319 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.375866207999998, 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.323 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.375841481999998, 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.324 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:11:54.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 32.375816907000001, 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:54.328 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:11:54.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:54.332 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:54.333 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:54.241+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:54.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:54.381 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:54.378+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:54.446 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:54.441+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:54.462 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:54.462 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:54.476 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:54.565 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:54.562+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:54.637 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:54.637 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:54.637 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:54.637 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:54.637 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:54.648 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:54.642+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:54.651 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:54.748 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:54.745+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:54.797 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.216550756000004, 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:11:54.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.216504088999997, 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:54.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.216430023000001, 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:54.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:11:54.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.216367297000005, 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:11:54.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:11:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:11:54.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.216315347000005, 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:11:54.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.842 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.216102261000003, 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.846 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.214218235999994, 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:11:54.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.214170186000004, 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:54.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:11:54.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.855 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.214121879000004, 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:11:54.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.214074663999995, 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:54.864 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.214010435000006, 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:11:54.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:11:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.213958524999995, 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.209356231000001, 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.209301416000002, 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:11:54.880 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.209253692000004, 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.881 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:11:54.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.209186802999994, 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:11:54.885 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:11:54.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:11:54.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.205641525000004, 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:11:54.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:11:54.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:11:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.205594783999999, 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:11:54.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.205548261999994, 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.898 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:11:54.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:11:54.902 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.410852083999998, 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:11:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:11:54.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:54.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:54.907 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:54.909 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:54.909 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:54.906+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:55.009 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:55.006+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:55.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:55.141 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:55.138+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:55.211 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:55.209+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:55.262 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:55.263 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:55.263 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:55.263 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:55.263 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:55.276 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:55.321 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:55.318+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:55.406 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 53.035737001000001, 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:11:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744474826999998, 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:11:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:11:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744445077999998, 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.74442397, 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:11:55.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:11:55.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744389089000002, 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:11:55.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:11:55.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744366679999999, 2024-09-21T23:11:55.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.484 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:55.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:11:55.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:11:55.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:55.490 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744345224, 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:11:55.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744314557999999, 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:11:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744294822000001, 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:11:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:11:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:11:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:11:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:11:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:11:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744274838000003, 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:11:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:11:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744256069000002, 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:11:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744236895999997, 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:11:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.744217976000002, 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:11:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:11:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:11:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.731108698, 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:11:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.731082831999998, 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:11:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:11:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.731048209000001, 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:11:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:11:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:11:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.731021554000002, 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:11:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:11:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.730998110000002, 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:11:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.730977903000003, 2024-09-21T23:11:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:11:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:11:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.730938723000001, 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:11:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:11:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:11:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:55.554 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:55.555 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:55.556 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:55.550+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:55.636 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:11:55.685 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:55.682+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:55.726 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:55.726 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:55.726 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:55.726 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:55.741 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:11:55.782 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:55.783 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:55.783 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:55.778+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:55.800 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:55.800 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:55.828 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:55.828 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:55.853 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:55.853 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:55.857 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:55.853+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:55.877 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:55.878 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:55.901 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:55.901 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:55.921 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:55.921 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:55.927 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:55.927 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:55.942 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:55.943 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:55.943 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:11:55.958 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:55.953+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:56.059 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:56.054+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:56.123 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:56.123 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:56.138 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:11:56.181 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:56.178+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:56.261 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:56.258+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:56.302 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:56.302 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:56.316 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:11:56.362 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:56.361+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:56.463 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:56.458+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:56.485 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:56.485 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:56.500 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:11:56.563 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:56.561+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:56.663 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:56.658+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:56.668 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:56.668 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:56.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:11:56.741 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:11:56.741 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:11:56.742 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.3558611148889147 2024-09-21T23:11:56.742 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 0.3558611148889147 2024-09-21T23:11:56.765 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:56.761+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:56.815 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:56.815 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:56.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:11:56.881 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:56.877+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:56.909 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:11:56.966 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:56.962+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:57.010 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:57.011 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:57.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:11:57.068 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:57.065+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:57.152 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:57.152 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:57.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:11:57.168 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:57.162+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:57.269 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:57.266+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:57.336 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:57.336 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:57.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:11:57.370 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:57.366+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:57.470 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:11:57.465+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:11:57.472 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:57.472 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:57.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:11:57.572 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:57.566+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:57.649 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:57.649 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:57.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:11:57.713 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:57.708+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:57.774 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:57.770+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:57.787 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:57.787 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:57.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:11:57.826 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:11:57.826 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.4 to 0.355861 (5b19) 2024-09-21T23:11:57.897 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:57.894+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:57.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:11:57.968 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:57.968 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:57.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:11:58.005 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:58.002+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:58.076 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:58.068+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:58.104 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:11:58.105 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:11:58.111 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:11:58.111 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:11:58.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:11:58.127 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:11:58.127 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:11:58.142 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:11:58.142 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:11:58.158 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:11:58.158 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:11:58.177 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:58.174+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:58.178 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:11:58.178 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:11:58.197 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:11:58.197 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:11:58.215 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:11:58.216 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:11:58.234 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:11:58.234 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:11:58.283 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:11:58.284 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:58.293 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:58.290+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:58.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:11:58.378 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:58.374+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:58.421 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:58.421 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:11:58.421 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:11:58.421 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:11:58.421 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:58.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:11:58.478 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:58.478+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:58.556 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 57.163339473000001, 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.557 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:11:58.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71501 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7200768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693394408000003, 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069501725, 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71501 2024-09-21T23:11:58.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860150+0000", 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-07 2024-09-21T23:11:58.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860152+0000", 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9870000000000002e-06 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860158+0000", 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860162+0000", 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9180000000000002e-06 2024-09-21T23:11:58.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929487+0000", 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069324330000000003 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929580+0000", 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2839999999999999e-05 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929651+0000", 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1416999999999997e-05 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.565 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693374603999999, 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:11:58.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:11:58.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:11:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:11:58.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:11:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:11:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693351448999998, 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:11:58.571 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:11:58.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.575 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693329231999996, 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:58.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:11:58.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693308621, 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693288997000003, 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.587 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693268060000001, 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.588 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:11:58.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:58.592 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693246096999999, 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693226672999998, 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:11:58.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:11:58.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:11:58.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693207532000002, 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.601 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693181082000002, 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693161562, 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.609 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693138756000003, 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.613 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:11:58.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:11:58.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693118284000001, 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.621 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693042519000002, 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.622 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:11:58.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:11:58.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:11:58.625 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.693018846999998, 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:11:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:11:58.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:11:58.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:11:58.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.688103675999997, 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.630 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:11:58.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.688078949999998, 2024-09-21T23:11:58.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 36.688054375, 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:11:58.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.642 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.643 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:11:58.643 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:11:58.643 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:11:58.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:11:58.644 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:11:58.638+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:11:58.745 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:58.740+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:58.776 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:58.776 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:58.776 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:58.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:11:58.846 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:58.842+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:58.918 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:58.919 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:58.933 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:11:58.979 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:11:58.974+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:11:59.048 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:11:59.040+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.501935497000005, 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.501888829999999, 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:11:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:11:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.501814764000002, 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.501752038000006, 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.501700088000007, 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:11:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:11:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:11:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:11:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:11:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:11:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:11:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:11:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:11:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:11:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:11:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:11:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:11:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:11:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.501487002000005, 2024-09-21T23:11:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:11:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:11:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:11:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:11:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:11:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:11:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:11:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:11:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:11:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:11:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:11:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:11:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:11:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:11:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:11:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:11:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:11:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.499602976999995, 2024-09-21T23:11:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:11:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:11:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:11:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.499554927000005, 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:11:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.499506620000005, 2024-09-21T23:11:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.499459404999996, 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:11:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.499395175999993, 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:11:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.499343265999997, 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:11:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:11:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.494740972000002, 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:11:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.494686157000004, 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:11:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:11:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:11:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:11:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.494638433000006, 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:11:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.494571543999996, 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.491026266000006, 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.490979525, 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:11:59.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:11:59.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:11:59.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:11:59.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.202 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:11:59.203 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:59.203 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.490933002999995, 2024-09-21T23:11:59.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:11:59.203 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:11:59.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.204 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:59.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:11:59.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:11:59.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:11:59.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:11:59.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:11:59.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:11:59.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:11:59.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:11:59.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:11:59.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:11:59.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:11:59.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:11:59.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:11:59.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:11:59.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:11:59.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.211 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.212 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:11:59.212 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:59.212 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.696236825, 2024-09-21T23:11:59.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:11:59.212 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:11:59.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.213 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:59.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:11:59.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:11:59.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:11:59.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:11:59.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:11:59.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:11:59.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:11:59.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:11:59.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:11:59.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:11:59.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:11:59.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:59.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:11:59.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:11:59.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:11:59.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:11:59.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:11:59.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:11:59.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:11:59.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:11:59.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.222 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.223 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:59.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:11:59.225 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:11:59.218+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:11:59.325 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:11:59.322+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:11:59.397 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:59.397 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:11:59.397 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:59.411 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:11:59.425 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:59.420+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:59.527 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:11:59.520+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:11:59.574 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:11:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:11:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:11:59.574 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:59.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:11:59.628 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:59.620+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:59.718 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:11:59.718 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:11:59.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:11:59.718 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:11:59.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.718 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.347376119000003, 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:11:59.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:11:59.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.056113945, 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.056084196, 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.056063088000002, 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:11:59.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:11:59.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:11:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.056028206999997, 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:11:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:11:59.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:11:59.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:11:59.741 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.056005798000001, 2024-09-21T23:11:59.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:11:59.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:11:59.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:11:59.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.748 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.055984342000002, 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:59.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:11:59.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.055953676000001, 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:11:59.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.055933940000003, 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.757 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.055913955999998, 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.761 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:11:59.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:11:59.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:11:59.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.055895186999997, 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:11:59.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.055876013999999, 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:11:59.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:11:59.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.055857093999997, 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.042747816000002, 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:11:59.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.042721950000001, 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:11:59.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:11:59.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.042687327000003, 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:11:59.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:11:59.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.042660671999997, 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.789 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:11:59.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:11:59.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:11:59.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.042637227999997, 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:11:59.795 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.796 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.796 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.796 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:11:59.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.797 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:59.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:59.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:11:59.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:11:59.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:11:59.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:11:59.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:11:59.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:11:59.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:11:59.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:11:59.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:11:59.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:11:59.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:11:59.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:11:59.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:11:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:11:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.804 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:11:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.042617020999998, 2024-09-21T23:11:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:11:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:11:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.806 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:11:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:11:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:11:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:11:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:11:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:11:59.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:11:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:11:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:11:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:11:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:11:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:11:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:11:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:11:59.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:11:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:11:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.813 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:11:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.042577841000004, 2024-09-21T23:11:59.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:11:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:11:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:11:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:11:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:11:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:11:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:11:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:11:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:11:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:11:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:59.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:11:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:11:59.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:11:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:11:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:11:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:11:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:11:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:11:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:11:59.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:11:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:11:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:11:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:11:59.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:11:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:11:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:11:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:11:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:11:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:11:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:11:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:11:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:11:59.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:11:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:11:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:11:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:11:59.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:11:59.824 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:11:59.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:11:59.827 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:59.820+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:11:59.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:11:59.926 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:11:59.920+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:00.004 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:00.005 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:00.026 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:00.022+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:00.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:00.132 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:00.130+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:00.222 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:00.222 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:00.222 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:00.222 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:00.222 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:00.228 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:00.222+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:00.244 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:00.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:00.333 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:00.330+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:00.382 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:12:00.431 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:00.424+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:00.474 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:00.474 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:00.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:00.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:00.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.475 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.475 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:00.475 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.979912917000007, 2024-09-21T23:12:00.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:00.475 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:00.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:00.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:00.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:00.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:00.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:00.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:00.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:00.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:00.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.483 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.979859478999998, 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:00.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:00.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:00.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:00.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:00.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:00.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:00.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:00.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:00.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:00.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:00.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:00.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:00.491 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.491 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.491 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.492 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.492 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:00.492 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.979808504000005, 2024-09-21T23:12:00.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:00.492 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:00.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:00.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:00.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:00.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:00.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:00.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:00.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:00.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:00.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:00.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:00.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:00.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:00.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:00.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:00.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:00.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:00.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:00.503 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.979757573000001, 2024-09-21T23:12:00.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:00.503 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.504 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:00.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.504 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:00.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:00.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:00.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:00.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:00.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:00.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.979702262000004, 2024-09-21T23:12:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.979633347000004, 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.974927852999997, 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.488774341999999, 2024-09-21T23:12:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.487546421000005, 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.487460626000001, 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.486819030999996, 2024-09-21T23:12:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.486744157000004, 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.486683378999999, 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.486647060999999, 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.682185266999994, 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 61.165647100999998, 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.055739020000001, 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.055707451, 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.055671448, 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.9288881409999998, 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:00.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:00.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:00.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.596 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:00.596 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:00.596 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:00.597 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:00.597 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:00.598 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:00.598 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:00.599 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:00.599 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:00.599 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:00.600 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:00.600 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:00.600 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:00.600 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:00.601 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:00.601 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:00.601 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:00.592+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:00.618 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:00.618 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:00.640 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:00.640 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:00.705 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:00.702+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:00.712 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:00.712 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:00.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:00.799 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:00.792+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:00.893 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:00.893 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:00.900 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:00.892+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:00.908 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:01.000 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:00.998+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:01.026 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:01.026 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:01.039 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:01.101 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:01.096+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:01.170 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:01.170 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:01.183 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:01.225 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:01.220+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:01.303 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:01.296+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:01.342 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:01.342 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:01.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:01.403 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:01.398+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:01.504 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:01.500+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:01.531 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:01.531 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:01.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:01.606 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:01.602+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:01.707 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:01.700+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:01.711 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:01.711 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:01.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:01.813 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:01.808+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:01.890 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:01.890 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:01.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:01.945 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:01.940+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:02.010 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:02.006+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:02.064 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:02.064 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:02.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:02.125 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:02.120+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:02.196 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:02.197 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:02.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:02.211 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:02.204+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:02.311 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:02.304+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:02.375 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:02.375 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:02.390 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:02.422 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:12:02.437 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:02.432+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:02.513 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:02.510+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:02.564 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:02.564 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:02.579 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:02.613 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:02.610+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:02.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:02.703 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:02.703 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:02.715 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:02.708+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:02.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:02.800 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:02.800 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:02.815 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:02.810+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:02.816 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:02.817 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:02.833 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:02.833 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:02.849 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:02.849 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:02.855 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:02.855 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:02.855 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T23:12:02.855 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:12:02.855 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:12:02.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T23:12:02.868 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:02.884 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:02.884 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:02.885 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:02.885 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:02.885 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:02.885 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:02.886 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:02.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:02.905 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:02.905 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:02.923 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:02.923 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:02.949 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:02.944+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:02.977 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:02.977 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:12:02.977 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:03.017 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:03.014+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.677276657, 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.069 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:12:03.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:12:03.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71501 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7200768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207331592000003, 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069501725, 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.074 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71501 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860149+0000", 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860150+0000", 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.58e-07 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860152+0000", 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9870000000000002e-06 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860158+0000", 2024-09-21T23:12:03.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860162+0000", 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9180000000000002e-06 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929487+0000", 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069324330000000003 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929580+0000", 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2839999999999999e-05 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929651+0000", 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1416999999999997e-05 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.078 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207311787999998, 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207288632999997, 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:12:03.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:12:03.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:12:03.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207266416000003, 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:12:03.090 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:12:03.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207245804999999, 2024-09-21T23:12:03.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:12:03.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.098 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207226181000003, 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:12:03.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:12:03.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207205244000001, 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:12:03.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:12:03.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207183280999999, 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.111 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207163856999998, 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:03.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207144716000002, 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:12:03.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:12:03.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207118266000002, 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:03.124 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207098746, 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:03.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:12:03.128 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207075940000003, 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:12:03.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:12:03.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.207055468, 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:12:03.133 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:12:03.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.137 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.206979703000002, 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:03.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:12:03.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.141 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.206956030999997, 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:03.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:12:03.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:12:03.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.145 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.202040859999997, 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:03.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:12:03.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.202016133999997, 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:12:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 41.201991559, 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:03.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:03.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:03.158 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:03.159 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:03.160 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:03.158+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:03.260 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:03.258+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:03.293 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:03.293 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:03.307 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:03.397 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:03.394+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:03.413 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:12:03.490 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:03.490 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:03.490 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:03.490 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:03.490 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:03.491 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:03.486+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:03.505 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:03.593 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:03.590+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:03.664 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:03.658+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:03.676 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.069958546999999, 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:03.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:12:03.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:12:03.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.069911880000006, 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:12:03.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:12:03.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.069837813999996, 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.685 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:12:03.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.689 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.069775088, 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:12:03.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:12:03.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.693 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.069723138000001, 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.694 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:12:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.069510051999998, 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:12:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.067626027000003, 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:12:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.067577976999999, 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.067529669999999, 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.712 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.067482455000004, 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:12:03.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:12:03.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.067418226000001, 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.720 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:12:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:03.724 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.067366316000005, 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:03.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:12:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.062764021999996, 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.729 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:12:03.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:12:03.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.062709206999997, 2024-09-21T23:12:03.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.062661482999999, 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.738 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:12:03.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:12:03.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:12:03.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.062594594000004, 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.742 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:12:03.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:03.746 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.059049315999999, 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:12:03.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.059002574999994, 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.751 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.058956053000003, 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.264259875, 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:12:03.759 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:12:03.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:03.764 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:03.766 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:03.766 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:03.762+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:03.867 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:03.864+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:03.940 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:03.940 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:03.940 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:03.940 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:03.955 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:04.001 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:03.998+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:04.068 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:04.064+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:04.077 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:04.078 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:04.092 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:04.169 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:04.164+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 61.891699717000002, 2024-09-21T23:12:04.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:12:04.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.267 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600437542999998, 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.268 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:12:04.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:04.272 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600407793999999, 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:12:04.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600386686, 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.277 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:12:04.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.281 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600351805000003, 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.285 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600329395999999, 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:12:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:12:04.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.60030794, 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:04.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600277274, 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.295 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:12:04.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600257538000001, 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.300 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:12:04.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600237554000003, 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:04.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:12:04.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:12:04.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600218785000003, 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:12:04.308 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600199611999997, 2024-09-21T23:12:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.600180692000002, 2024-09-21T23:12:04.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:12:04.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.587071414, 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.587045547999999, 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.587010925000001, 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.586984270000002, 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.336 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.586960826000002, 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:12:04.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:04.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:12:04.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:12:04.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.586940619000003, 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:12:04.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.586901439000002, 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:12:04.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:12:04.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.349 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:04.350 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:04.351 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:04.348+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:04.451 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:04.446+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:04.490 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:04.490 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:04.490 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:04.490 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:04.507 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:04.593 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:04.590+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:04.653 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:04.650+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:04.677 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:04.677 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:04.692 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:04.753 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:04.752+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:04.854 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:04.850+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.363350269999998, 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:04.858 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:04.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:04.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.363296832000003, 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.363245856999995, 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.870 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.363194926000006, 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:04.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:04.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.363139614999994, 2024-09-21T23:12:04.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:04.879 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.363070699999994, 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:04.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.883 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.358365206000002, 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:04.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.872211695000004, 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.870983773999995, 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:04.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.870897979000006, 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:04.900 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.870256384000001, 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.870181509999995, 2024-09-21T23:12:04.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:04.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.870120732000004, 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:04.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.870084414000004, 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.912 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:04.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:04.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:04.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.065622619999999, 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.916 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 65.549084453999996, 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:04.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.439176373, 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:04.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.439144804, 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:04.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:04.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:04.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:04.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.439108801, 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:04.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:04.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.312325494, 2024-09-21T23:12:04.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:04.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:04.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:04.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:04.947 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:04.948 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:04.949 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:04.994 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:04.990+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:05.055 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:05.050+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:05.126 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:05.127 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:05.127 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:05.127 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:05.137 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:05.138 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:05.141 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:05.154 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:05.155 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:05.157 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:05.154+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:05.173 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:05.173 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:05.189 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:05.189 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:05.210 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:05.210 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:05.230 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:05.231 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:05.251 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:05.251 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:05.258 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:05.254+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:05.269 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:05.269 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:05.272 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:05.285 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:05.359 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:05.356+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:05.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:12:05.459 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:05.456+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:05.473 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:05.473 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:05.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:05.560 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:05.556+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:05.630 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:05.630 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:05.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:05.689 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:05.688+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:05.762 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:05.758+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:05.808 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:05.808 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:05.823 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:05.863 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:05.858+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:05.946 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:05.946 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:05.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:05.964 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:05.958+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:06.065 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:06.064+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:06.082 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:06.082 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:06.096 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:06.165 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:06.162+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:06.261 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:06.261 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:06.267 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:06.262+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:06.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:06.368 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:06.362+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:06.392 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:06.392 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:06.406 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:06.437 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:12:06.468 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:06.464+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:06.541 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:06.541 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:06.556 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:06.601 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:06.600+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:06.669 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:06.668+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:06.724 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:06.724 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:06.739 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:06.771 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:06.766+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:06.871 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:06.866+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:06.900 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:06.900 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:06.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:06.972 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:06.966+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:07.032 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:07.032 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:07.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:07.093 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:07.092+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:07.167 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:07.167 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:07.173 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:07.170+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:07.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:07.274 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:07.272+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:07.292 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:07.346 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:07.346 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:07.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:07.409 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:07.406+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:07.455 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:07.455 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:07.474 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:07.470+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:07.478 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:07.478 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:07.493 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:07.494 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:07.499 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:07.499 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:07.511 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:07.519 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:07.519 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:07.544 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:07.544 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:07.566 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:07.566 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:07.575 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:07.572+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:07.588 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:07.588 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:07.609 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:07.609 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:07.657 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:07.705 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:07.702+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:07.777 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:07.774+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.216467402000006, 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:12:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:12:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.216420735, 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:12:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.216346669000004, 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:12:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:12:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:12:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.216283942999993, 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:12:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:12:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.216231992999994, 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:12:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:12:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.216018907000006, 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:12:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.214134881999996, 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.214086832000007, 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.214038525000007, 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:12:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.213991309999997, 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:12:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:12:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:12:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:12:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:12:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.213927080999994, 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:12:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.213875170999998, 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.209272877000004, 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:12:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:12:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:12:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.209218062000005, 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:12:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:12:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.209170338000007, 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:12:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:12:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:12:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.209103448999997, 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.205558171000007, 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:12:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.205511430000001, 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:12:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:12:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.205464907999996, 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:12:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.410768730000001, 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:07.912 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:07.913 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:07.914 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:07.912+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:07.992 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:07.993 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:07.993 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:12:07.993 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-21T23:12:07.993 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-21T23:12:08.014 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:08.012+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:08.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:08.152 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:08.150+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:08.216 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:08.210+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:08.222 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:08.223 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:08.236 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:08.321 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:08.318+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:08.401 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 66.032621865999999, 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:12:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:12:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:12:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741359692000003, 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:12:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:12:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741329942999997, 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741308834999998, 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:12:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741273954, 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741251544999997, 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741230088999998, 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:12:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741199422999998, 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741179686999999, 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:12:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741159703000001, 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741140934000001, 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741121761000002, 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.741102841, 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:12:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.727993562999998, 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.727967696999997, 2024-09-21T23:12:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.727933073999999, 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:12:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.727906419, 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:12:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.727882975, 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.727862768000001, 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:12:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:12:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 45.727823588, 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:12:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:12:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:08.488 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:12:08.489 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:08.490 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-21T23:12:08.490 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:08.486+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:08.589 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:08.588+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:08.662 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:08.662 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:08.662 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:08.662 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:08.676 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:08.690 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:08.688+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:08.782 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:08.782 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:08.782 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:08.782 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:08.782 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:08.794 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:08.788+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:08.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:08.892 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:08.886+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.426976592000003, 2024-09-21T23:12:08.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:08.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:08.925 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.426923153999994, 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:08.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:08.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.929 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.426872179, 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:08.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.933 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.426821247999996, 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:08.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.426765936999999, 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.938 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.426697021999999, 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:08.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:08.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.421991528000007, 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.935838016999995, 2024-09-21T23:12:08.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.954 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.934610096, 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:08.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:08.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.958 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.934524300999996, 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:08.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:08.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:08.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.962 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.933882706000006, 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.933807831999999, 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.933747053999994, 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.933710735999995, 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.129248942000004, 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.979 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:08.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 69.612710776, 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.502802695, 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.502771125999999, 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:08.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:08.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:08.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:08.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.502735123000001, 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:09.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:09.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.375951816000001, 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.005 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:09.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:09.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:09.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.010 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:09.011 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:09.011 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:09.008+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:09.111 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:09.108+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:09.180 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:09.180 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:09.180 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:09.180 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:09.194 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:09.241 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:09.238+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:09.311 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:09.311 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:09.311 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:09.311 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:09.311 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:09.312 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:09.312+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:09.327 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:09.414 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:09.410+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:09.453 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:12:09.497 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:09.497 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:09.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:09.497 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:09.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.421240864, 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.165689368999999, 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:09.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.165665148999999, 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:09.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:09.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.165618762999998, 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:09.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:09.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:09.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.165603539999999, 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:09.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77015 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.154484744000001, 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059342726999999998, 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77015 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341443+0000", 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341445+0000", 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341453+0000", 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4869999999999996e-06 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341458+0000", 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:09.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400574+0000", 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059115684000000002 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400680+0000", 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010637399999999999 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400785+0000", 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001047 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.522 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.154436785999998, 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.523 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.154414058999997, 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:09.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:09.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.530 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.154385462, 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:09.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.154373258, 2024-09-21T23:12:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.154354267000002, 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:09.538 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:09.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.154019951000002, 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:09.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.153975825000003, 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.153951868, 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:09.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.153911665000003, 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:09.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.142772475000001, 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:09.558 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:09.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.142750521000004, 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.562 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:09.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.142724102999999, 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.142697806999998, 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:09.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:09.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 47.142675195000002, 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:09.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:09.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:09.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.578 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:09.578 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:09.578 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:09.579 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:09.579 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:09.574+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:09.628 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:09.679 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:09.676+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:09.723 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:09.723 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:09.723 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:09.723 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:09.737 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:09.780 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:09.774+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:09.783 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:09.783 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:09.799 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:09.816 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:09.816 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:09.832 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:09.832 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:09.851 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:09.851 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:09.871 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:09.871 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:09.881 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:09.878+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:09.891 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:09.891 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:09.901 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:09.902 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:09.912 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:09.912 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:09.916 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:09.982 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:09.980+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:10.032 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:10.032 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:10.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:10.083 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:10.078+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:10.183 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:10.178+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:10.207 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:10.207 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:10.223 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:10.284 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:10.284+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:10.344 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:10.345 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:10.358 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:10.385 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:10.382+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:10.486 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:10.482+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:10.536 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:10.536 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:10.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:10.586 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:10.582+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:10.683 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:10.687 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:10.682+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:10.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:10.789 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:10.786+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:10.821 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:10.821 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:10.835 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:10.889 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:10.886+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:10.960 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:10.960 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:10.974 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:10.989 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:10.986+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:11.090 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:11.088+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:11.092 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:11.092 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:11.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:11.193 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:11.192+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:11.270 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:11.270 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:11.284 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:11.329 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:11.328+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:11.392 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:11.386+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:11.455 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:11.455 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:11.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:11.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:12:11.517 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:11.514+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:11.594 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:11.590+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:11.608 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:11.608 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:11.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:11.695 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:11.692+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:11.782 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:11.782 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:11.796 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:11.790+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:11.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:11.897 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:11.894+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:11.930 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:11.960 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:11.960 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:11.975 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:12.021 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:12.018+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:12.076 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:12.076 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:12.094 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:12.094 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:12.097 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:12.094+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:12.111 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:12.127 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:12.128 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:12.139 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:12.139 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:12.139 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:12.139 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:12.148 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:12.149 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:12.152 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:12.169 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:12.170 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:12.191 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:12.191 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:12.199 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:12.196+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:12.212 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:12.212 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:12.289 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:12.289 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:12.290 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:12.290 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:12.290 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:12.300 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:12.298+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:12.303 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:12.401 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:12.400+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:12.470 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.098545130000005, 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:12:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807282956000002, 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:12:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807253207000002, 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:12:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807232098999997, 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807197217999999, 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807174809000003, 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:12:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807153352999997, 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807122687000003, 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:12:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:12:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807102950999997, 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:12:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807082966999999, 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807064197999999, 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807045025000001, 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.807026104999998, 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:12:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.793916826999997, 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.793890961000002, 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:12:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.793856337999998, 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:12:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:12:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.793829682999998, 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:12:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.793806238999998, 2024-09-21T23:12:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:12:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:12:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.793786032, 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 49.793746851999998, 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:12:12.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:12:12.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:12:12.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.559 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:12.559 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:12.559 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:12.560 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:12:12.561 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:12.562 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:12.554+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:12.661 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:12.660+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:12.737 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:12.737 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:12.737 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:12.737 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:12.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:12.762 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:12.760+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:12.863 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:12.860+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:12.871 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:12.871 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:12.871 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:12.871 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:12.871 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:12.885 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:12.964 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:12.958+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:13.009 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.515158182999997, 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:13.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:13.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.515104745000002, 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:13.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.515053769999994, 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.018 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:13.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:13.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.021 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.515002839000005, 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:13.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.514947527999993, 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.514878612999993, 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.510173119000001, 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.037 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.024019608000003, 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:13.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.041 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.022791686999994, 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.022705892000005, 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:13.045 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:13.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.022064297, 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:13.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.021989422999994, 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:13.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.021928645000003, 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:13.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:13.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:13.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.021892327000003, 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:13.065 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.217430532999998, 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:13.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:13.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:13.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:13.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 73.700892366999994, 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:13.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.590984286000001, 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:13.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.590952717, 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.590916713999999, 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.464133406999998, 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.097 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:13.098 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:13.098 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:13.096+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:13.198 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:13.196+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:13.267 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:13.268 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:13.268 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:13.268 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:13.282 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:13.329 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:13.326+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:13.400 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:13.400 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:13.400 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:13.400 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:13.400 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:13.400 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:13.396+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:13.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:13.490 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:13.491 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:13.491 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-21T23:12:13.491 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:12:13.491 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:12:13.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-21T23:12:13.501 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:13.498+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.517551, 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.595 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:13.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:13.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:13.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.261999504999999, 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.605 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.261975284999998, 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:13.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:13.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.609 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.261928898999997, 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:13.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.261913675999999, 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:13.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77015 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250794880000001, 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059342726999999998, 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77015 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341443+0000", 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341445+0000", 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341453+0000", 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4869999999999996e-06 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341458+0000", 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400574+0000", 2024-09-21T23:12:13.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059115684000000002 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400680+0000", 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010637399999999999 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400785+0000", 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001047 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250746921999998, 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:13.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250724194999997, 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:13.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250695598, 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:13.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250683393999999, 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:13.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:13.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250664403000002, 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:13.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250330087000002, 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:13.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:13.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.648 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250285961000003, 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:13.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:13.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250262004, 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.653 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:13.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.250221801000002, 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:13.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:13.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:13.662 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.239082611000001, 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:13.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.667 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.239060657000003, 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.671 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.239034238999999, 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.672 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:13.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.239007942999997, 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:13.676 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.238985331000002, 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:13.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:13.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:13.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:13.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:13.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:13.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:13.685 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:13.687 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:13.687 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "" 2024-09-21T23:12:13.687 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:13.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:13.688 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:13.684+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:13.787 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:13.784+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:13.855 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:13.855 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:13.855 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:13.855 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:13.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:13.889 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:13.886+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:13.989 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:13.986+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:14.031 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:14.031 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:14.031 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:14.032 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:14.032 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:14.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:14.090 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:14.086+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:14.166 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.340348935999998, 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.340277927999999, 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.171 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:14.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.340213636999998, 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.176 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:14.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:14.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:14.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.340183138, 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.180 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.340035151999999, 2024-09-21T23:12:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.188 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339978647000002, 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.189 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339924072000002, 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339868854000002, 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.198 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:14.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:14.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:14.201 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339814224999998, 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.202 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339695075000002, 2024-09-21T23:12:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.210 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339617640999997, 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.211 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:14.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339552019000003, 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.215 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:14.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:14.219 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339487691000002, 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:14.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:14.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:14.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339426603, 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:14.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:14.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339366138000003, 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:14.228 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.232 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339301939999999, 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.233 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339227932, 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339190506000001, 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:14.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:14.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:14.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339143984000003, 2024-09-21T23:12:14.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 61.339084489000001, 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:14.254 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:14.254 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:14.254 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:14.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:14.256 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:14.250+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:14.356 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:14.352+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:14.382 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:14.383 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:14.383 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:14.383 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:14.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:14.414 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:14.414 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:14.433 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:14.433 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:14.450 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:14.456 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:14.450+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:14.468 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:14.469 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:14.487 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:14.487 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:14.518 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:14.519 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:14.541 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:14.541 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:14.557 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:14.556+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:14.561 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:14.562 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:14.562 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:12:14.576 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:14.576 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:14.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:14.658 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:14.654+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:14.758 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:14.758 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:14.759 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:14.756+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:14.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:14.859 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:14.854+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:14.937 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:14.938 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:14.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:14.961 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:14.958+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:15.062 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:15.058+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:15.066 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:15.066 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:15.080 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:15.163 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:15.162+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:15.197 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:15.197 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:15.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:15.263 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:15.258+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:15.329 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:15.329 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:15.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:15.389 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:15.388+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:15.396 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:12:15.461 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:15.461 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:15.469 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:15.466+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:15.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:15.566 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:15.562+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:15.636 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:15.636 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:15.649 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:15.667 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:15.664+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:15.767 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:15.764+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:15.771 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:15.771 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:15.784 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:15.868 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:15.864+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:15.906 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:15.906 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:15.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:15.973 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:15.970+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:16.042 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:16.042 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:16.055 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:16.070 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:16.068+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:16.171 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:16.168+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:16.213 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:16.214 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:16.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:16.272 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:16.272+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:16.348 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:16.348 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:16.362 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:16.413 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:16.410+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:16.475 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:16.472+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:16.484 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:16.484 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:16.498 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:16.582 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:16.585 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:16.582+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:16.664 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:16.664 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:16.677 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:16.674+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:16.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:16.728 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:16.747 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:16.747 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:16.764 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:16.764 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:16.778 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:16.774+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:16.780 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:16.780 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:16.801 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:16.801 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:16.822 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:16.822 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:16.841 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:16.841 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:16.841 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:16.841 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:16.841 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:16.842 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:16.843 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:16.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:16.864 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:16.864 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:16.879 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:16.876+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:16.980 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:16.976+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:16.981 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.487345665999996, 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:16.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:16.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:16.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.487292228000001, 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.986 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.487241252999993, 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:16.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.487190322000004, 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:16.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:16.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:16.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.487135011000007, 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:16.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.487066096000007, 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:17.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.482360602, 2024-09-21T23:12:17.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.996207091000002, 2024-09-21T23:12:17.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:17.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.994979169999993, 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:17.013 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:17.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.994893375000004, 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:17.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.994251779999999, 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:17.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:17.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.994176906000007, 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:17.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:17.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.994116128000002, 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.994079810000002, 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.189618015999997, 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.037 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:17.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.673079849999993, 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:17.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 26.563171769, 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:17.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:17.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.050 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 26.563140199999999, 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.051 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:17.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:17.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 26.563104197000001, 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.056 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:17.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:17.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 26.436320890000001, 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:17.061 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:17.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.066 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:17.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:17.113 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:17.109+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:17.181 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:17.177+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:17.193 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:17.193 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:17.193 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:17.193 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:17.207 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:17.293 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:17.289+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:17.371 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:17.371 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:17.371 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:17.371 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:17.371 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:17.383 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:17.377+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:17.385 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:17.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:12:17.484 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:17.480+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:17.567 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:17.567 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:17.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.491615509, 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.568 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:17.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.573 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.236064014, 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:17.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.236039794, 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:17.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:17.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.235993407999999, 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:17.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:17.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.586 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.235978185, 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:17.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77015 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224859389000002, 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059342726999999998, 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77015 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341443+0000", 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341445+0000", 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341453+0000", 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4869999999999996e-06 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341458+0000", 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400574+0000", 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059115684000000002 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400680+0000", 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010637399999999999 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400785+0000", 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001047 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224811430999999, 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:17.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224788703999998, 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.599 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:17.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:17.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224760107000002, 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.603 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:17.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:17.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:17.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224747903000001, 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:17.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.611 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224728912000003, 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:17.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:17.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:17.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224394596000003, 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.616 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224350469999997, 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.620 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:17.624 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224326513000001, 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.625 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.224286309999997, 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:17.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:17.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:17.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.213147120000002, 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:17.633 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.213125165999998, 2024-09-21T23:12:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.641 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.213098748, 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.213072451999999, 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.646 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.213049839999996, 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:17.650 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:17.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:17.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.654 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:17.654 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:17.654 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:17.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:17.656 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:17.652+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:17.756 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:17.749+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:17.786 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:17.786 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:17.786 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:17.786 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:17.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:17.857 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:17.853+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:17.921 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:17.922 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:17.922 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:17.922 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:17.922 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:17.936 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:17.957 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:17.953+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:18.058 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.231461280000005, 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:18.062 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.231390271999999, 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.063 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.231325980999998, 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.067 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.231295481999993, 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.231147496000006, 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:18.075 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:18.079 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.231090991000002, 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:18.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:18.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.083 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.231036415999995, 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:18.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230981197999995, 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230926569000005, 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230807419000001, 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:18.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:18.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230729984999996, 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.100 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:18.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230664363000002, 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:18.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230600034999995, 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.108 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:18.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:18.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230538946999999, 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.112 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:18.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:18.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230478481999995, 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230414284000005, 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230340276000007, 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:18.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230302850000001, 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:18.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230256327999996, 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.132 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:18.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:18.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:18.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 65.230196832999994, 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.136 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.140 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:18.141 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:18.053+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:18.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:18.158 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:18.153+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:18.259 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:18.256+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:18.267 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:18.267 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:18.267 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:18.267 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:18.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:18.360 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:18.353+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:18.447 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:18.447 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:18.447 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:18.447 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:18.447 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:18.461 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:18.457+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:18.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:18.535 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:12:18.562 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:18.560+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:18.598 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.868140206999996, 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:12:18.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:12:18.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:12:18.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:12:18.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.604 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220826713999998, 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:12:18.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.608 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220794038999998, 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:12:18.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220770704000003, 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.616 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220738138999998, 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.617 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:12:18.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:12:18.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:12:18.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:12:18.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:18.622 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220650517000003, 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:12:18.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220531770000001, 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:12:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:12:18.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220507005000002, 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.631 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:12:18.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220485858000004, 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:12:18.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:12:18.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220428161000001, 2024-09-21T23:12:18.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:12:18.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.644 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220393280000003, 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:12:18.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.648 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220367754999998, 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220296568999998, 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:12:18.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220260723000003, 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.661 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220243629999999, 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:18.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:12:18.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220227618999999, 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220170371000002, 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:12:18.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:12:18.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220089006000002, 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.674 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:12:18.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:18.678 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.220057455999999, 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 56.219910558999999, 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:18.683 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:12:18.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:12:18.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:12:18.686 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:18.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:18.687 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:18.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:18.689 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:18.684+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:18.689 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:18.689 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:12:18.788 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:18.788+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:18.863 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:18.863 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:18.863 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:18.863 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:18.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:18.882 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:18.929 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:18.925+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:18.991 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:18.985+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:19.027 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:19.027 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:19.043 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:19.044 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:19.044 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:19.045 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:19.045 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:19.045 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:19.045 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:19.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:19.060 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:19.061 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:19.076 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:19.077 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:19.092 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:19.085+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:19.097 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:19.097 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:19.116 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:19.116 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:19.135 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:19.136 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:19.156 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:19.156 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:19.183 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:19.183 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:19.193 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:19.192+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:19.198 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:19.293 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:19.292+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:19.364 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:19.365 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:19.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:19.395 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:19.389+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:19.496 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:19.492+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:19.541 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:19.541 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:19.553 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:12:19.561 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:19.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:12:19.605 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:19.604+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:19.697 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:19.696+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:19.735 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:19.735 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:19.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:19.798 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:19.793+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:19.870 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:19.870 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:19.885 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:19.900 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:19.896+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:20.000 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:19.996+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:20.054 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:20.054 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:20.068 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:20.102 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:20.100+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:20.190 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:20.190 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:20.203 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:20.197+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:20.204 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:20.303 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:20.297+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:20.366 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:20.366 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:20.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:20.387 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:12:20.405 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:20.404+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:20.419 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:12:20.505 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:20.501+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:20.523 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:20.523 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:20.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:20.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:12:20.607 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:20.604+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:20.700 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:20.700 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:20.707 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:20.701+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:20.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:20.809 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:20.805+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:20.867 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:20.867 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:20.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:20.921 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:20.917+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:21.010 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:21.005+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:21.030 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:21.030 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:21.043 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:21.111 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:21.108+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:21.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:21.204 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:21.204 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:21.212 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:21.208+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:21.218 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:21.312 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:21.312 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:21.312 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:21.309+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:21.329 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:21.329 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:21.345 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:21.345 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:21.362 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:21.362 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:21.376 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:21.376 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:21.376 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:21.376 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:21.382 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:21.382 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:21.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:21.403 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:21.404 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:21.423 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:21.424 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:21.424 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:21.417+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:21.444 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:21.444 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:21.509 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:21.509 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:21.509 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:21.514 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:21.509+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:21.521 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:21.542 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:12:21.554 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:12:21.580 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:12:21.625 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:21.621+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:21.679 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:21.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.602887796, 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:21.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.685 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.347336300999999, 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:21.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:21.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.347312080999998, 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.690 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:21.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:21.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.347265694999997, 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.347250471999999, 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:21.698 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:21.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77015 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.336131676000001, 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059342726999999998, 2024-09-21T23:12:21.702 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77015 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341442+0000", 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341443+0000", 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341445+0000", 2024-09-21T23:12:21.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341453+0000", 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4869999999999996e-06 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341458+0000", 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400574+0000", 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059115684000000002 2024-09-21T23:12:21.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400680+0000", 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010637399999999999 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400785+0000", 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0001047 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.336083717999998, 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:21.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:21.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.336060990999997, 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.711 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.336032394, 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.715 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.336020189999999, 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.336001199000002, 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.335666883000002, 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:21.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:21.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.335622757000003, 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.731 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.3355988, 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.335558597000002, 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.739 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:21.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:21.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.324419407000001, 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.743 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:21.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:21.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:21.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:21.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.748 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.324397453000003, 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:21.753 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.324371034999999, 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:21.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:21.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.758 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.324344738999997, 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:21.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:21.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:21.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.324322127000002, 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:21.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:21.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:21.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:21.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:21.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:21.769 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:21.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:21.771 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:21.765+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:21.871 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:21.868+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:21.903 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:21.903 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:21.903 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:21.903 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:21.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:21.972 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:21.969+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:22.073 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:22.069+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:22.081 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:22.081 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:22.081 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:22.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:22.173 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:22.172+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388721224999998, 2024-09-21T23:12:22.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:22.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:22.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388650217000006, 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388585926000005, 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.227 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388555427, 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.228 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:22.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:22.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:22.231 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388407440999998, 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.232 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:22.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:22.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:22.235 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388350935999995, 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:22.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.239 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388296361000002, 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:22.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388241143000002, 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.244 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:22.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388186513999997, 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.248 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.252 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.388067363999994, 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.256 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387989930000003, 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:22.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:22.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387924307999995, 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.261 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387859980000002, 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387798892000006, 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.269 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:22.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:22.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387738427000002, 2024-09-21T23:12:22.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:22.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.277 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387674228999998, 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.281 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387600221, 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:22.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:22.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:22.285 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387562794999994, 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.286 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:22.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387516273000003, 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.290 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:22.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.387456778000001, 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:22.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.298 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:22.299 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:22.300 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:22.293+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:22.400 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:22.397+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:22.468 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:22.468 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:22.469 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:22.469 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:22.484 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:22.529 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:22.528+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:22.566 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:12:22.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:12:22.616 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:22.616 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:22.616 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:22.616 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:22.616 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:22.632 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:22.641 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:22.637+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:22.725 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:22.724+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:22.802 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:22.800+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:22.804 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 104.073979974, 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.805 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:12:22.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:12:22.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:12:22.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426666480999998, 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:12:22.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426633805999998, 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426610470999996, 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.820 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:12:22.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426577905999999, 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.824 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:12:22.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426490284000003, 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:22.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:12:22.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426371537000001, 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.832 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426346772000002, 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.836 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426325624999997, 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:12:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426267928000001, 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:12:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:12:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:12:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426233046999997, 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:12:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426207521999999, 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:12:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:12:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426136335999999, 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426100490000003, 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:12:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:12:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426083396999999, 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:12:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426067386, 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.426010138000002, 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.425928773000003, 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:12:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.425897223, 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 60.425750325999999, 2024-09-21T23:12:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:22.888 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:22.889 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:22.933 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:22.932+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:23.005 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:23.004+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:23.059 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:23.059 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:23.059 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:23.059 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:23.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:23.117 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:23.116+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:23.207 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:23.204+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:23.242 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:23.242 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:23.242 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:23.242 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:23.242 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:23.256 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:23.308 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:23.305+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:23.409 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:23.405+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:23.426 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:23.426 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:23.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:23.426 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:23.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.426 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.23922702799999, 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.430 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.23912394199999, 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:12:23.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.433 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.23905659299999, 2024-09-21T23:12:23.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:12:23.437 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16386965699999, 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.438 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:23.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:12:23.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.441 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16381388900001, 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.442 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:12:23.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:12:23.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16378234999999, 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:12:23.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.449 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16374960100001, 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:12:23.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.452 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16372270799999, 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.454 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:12:23.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16369743499999, 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.458 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:23.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:12:23.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.461 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16366630799999, 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.462 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:23.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:12:23.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.465 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.16302295200001, 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.466 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15783702499999, 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:12:23.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:12:23.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:23.473 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15781554200001, 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:12:23.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.157791256, 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.477 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:12:23.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15777358899999, 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.481 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:12:23.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15775000400001, 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.485 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:23.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.488 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15773398900001, 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:12:23.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.492 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15772015799999, 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:23.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:12:23.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15768879699999, 2024-09-21T23:12:23.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:12:23.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:12:23.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 136.15762037499999, 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.500 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:12:23.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:12:23.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.504 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:23.504 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:23.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:23.549 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:23.548+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:23.567 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:12:23.574 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:12:23.610 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:12:23.610 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:23.608+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123241 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:23.606363+0000", 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.034503006000000003, 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034541385000000001, 2024-09-21T23:12:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for peered", 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123241 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606363+0000", 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606363+0000", 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606365+0000", 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8840000000000001e-06 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606373+0000", 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9699999999999999e-06 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606392+0000", 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.91e-05 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606402+0000", 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6760000000000002e-06 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606527+0000", 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125473 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:23.606543+0000", 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.552e-05 2024-09-21T23:12:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:12:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:12:23.645 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:23.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:23.686 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:23.686 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:23.705 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:23.706 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:23.711 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:23.705+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:23.724 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:23.725 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:23.755 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:23.756 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:23.774 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:23.774 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:23.795 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:23.795 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:23.811 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:23.805+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:23.814 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:23.814 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:23.817 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:23.817 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:23.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:23.832 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:23.832 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:23.912 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:23.908+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:23.949 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:23.950 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:23.964 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:24.013 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:24.009+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:24.078 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:24.078 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:24.092 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:24.114 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:24.112+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:24.215 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:24.212+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:24.250 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:24.250 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:24.263 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:24.316 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:24.309+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:24.383 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:24.384 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:24.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:24.417 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:24.416+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:24.518 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:24.513+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:24.559 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:24.559 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:24.571 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:24.582 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:12:24.583 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:12:24.607 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T23:12:24.653 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:24.649+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:24.703 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:24.703 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:24.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:24.719 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:24.716+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:24.820 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:24.817+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:24.824 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:24.824 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:24.837 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:24.924 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:24.921+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:24.988 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:24.988 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:25.001 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:25.049 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:25.045+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:25.124 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:25.121+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:25.163 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:25.163 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:25.175 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:25.260 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:25.257+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:25.326 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:25.324+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:25.335 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:25.336 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:25.350 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:25.401 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T23:12:25.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T23:12:25.434 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:25.424+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:25.528 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:25.524+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:25.529 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:25.529 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:25.545 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:25.629 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:25.628+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:25.674 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:25.675 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:25.688 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:25.733 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:25.729+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:25.801 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:25.801 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:25.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:25.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:25.861 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:25.860+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:25.931 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:25.925+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:25.947 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:25.947 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:25.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:26.005 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:26.005 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:26.022 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:26.022 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:26.031 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:26.025+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:26.040 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:26.041 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:26.057 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:26.057 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:26.076 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:26.076 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:26.096 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:26.096 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:26.118 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:26.168 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:26.169 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:26.169 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:26.170 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:26.170 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:26.171 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:26.171 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:26.171 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:26.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:26.217 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:26.213+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:26.271 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:26.265+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479694291000001, 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479623282999995, 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.311 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:26.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479558991999994, 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.315 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479528493000004, 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:26.319 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:26.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:26.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479380507000002, 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.323 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:26.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:26.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479324001999998, 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.327 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479269427000006, 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.331 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479214209000006, 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.335 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:26.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479159580000001, 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.339 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.479040429999998, 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.346 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478962996000007, 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:26.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.350 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478897373999999, 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:26.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478833046000005, 2024-09-21T23:12:26.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:26.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:26.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478771957999996, 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.358 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478711493000006, 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478647295000002, 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:26.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.369 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478573287000003, 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.370 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.373 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478535860999997, 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:26.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.377 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478489339000006, 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:26.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:26.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 73.478429844000004, 2024-09-21T23:12:26.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.385 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:26.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:26.387 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:26.384+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:26.488 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:26.481+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPushReply(3.a 144/143 [PushReplyOp(3:5063f514:::rbd_data.1149d3067c2b.0000000000000002:head)])", 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.578908+0000", 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0023349310000000002, 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023743169999999999, 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.578908+0000", 2024-09-21T23:12:26.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.578908+0000", 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.578910+0000", 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6029999999999999e-06 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.578920+0000", 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.013e-05 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.578950+0000", 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0549999999999997e-05 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.578965+0000", 2024-09-21T23:12:26.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5096000000000001e-05 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.579046+0000", 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0365999999999996e-05 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:12:26.587 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:26.588 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:26.585+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:26.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:26.607 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:12:26.619 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T23:12:26.651 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:12:26.651 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:12:26.689 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:26.688+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:26.735 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:26.736 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:26.736 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:26.736 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:26.736 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:26.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:26.797 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:26.796+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:26.891 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:26.888+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:26.928 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 108.197686936, 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.929 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:26.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:12:26.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550373442999998, 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.935 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:12:26.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:12:26.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550340767999998, 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.939 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:26.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:12:26.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:12:26.942 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550317433000004, 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.943 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:12:26.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:12:26.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.946 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550284868000006, 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.947 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550197245999996, 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:12:26.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:12:26.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.954 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550078498999994, 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:26.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:12:26.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:26.958 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550053734000002, 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:12:26.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:26.962 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.550032587000004, 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:12:26.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549974890000001, 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:12:26.966 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:12:26.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549940008999997, 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.970 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:26.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:12:26.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:12:26.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549914483999999, 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.974 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:26.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:12:26.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549843297999999, 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549807451999996, 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:12:26.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549790358999999, 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.986 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549774348, 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.990 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549717099999995, 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:26.994 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:26.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:12:26.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:26.999 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549635734999995, 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:27.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:12:27.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549604185000007, 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.007 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 64.549457287999999, 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.008 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:12:27.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:12:27.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:12:27.011 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.012 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.012 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.012 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.012 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:27.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:27.013 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:27.012+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:27.018 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:12:27.019 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:12:27.038 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-21T23:12:26.608678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1024950986,"num_objects":1650,"num_object_clones":294,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":963,"num_read":178579,"num_read_kb":834299,"num_write":21275,"num_write_kb":402032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3568,"num_bytes_recovered":228071938,"num_keys_recovered":600,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19067,"ondisk_log_size":19067,"up":82,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":768888,"kb_used_data":406724,"kb_used_omap":25,"kb_used_meta":361958,"kb_avail":754205832,"statfs":{"total":773094113280,"available":772306771968,"internally_reserved":0,"allocated":416485376,"data_stored":434886262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":370645039},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-107280653,"num_objects":-84,"num_object_clones":-34,"num_object_copies":-168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":-84,"num_whiteouts":-19,"num_read":721,"num_read_kb":616,"num_write":122,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1199468,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.041663"},"pg_stats":[{"pgid":"3.1f","version":"136'557","reported_seq":3959,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400095+0000","last_change":"2024-09-21T23:12:23.575904+0000","last_active":"2024-09-21T23:12:25.400095+0000","last_peered":"2024-09-21T23:12:25.400095+0000","last_clean":"2024-09-21T23:12:25.400095+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:12:25.400095+0000","last_undegraded":"2024-09-21T23:12:25.400095+0000","last_fullsized":"2024-09-21T23:12:25.400095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:06:35.248845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017863670000000001,"stat_sum":{"num_bytes":24955582,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":30,"num_read":6330,"num_read_kb":19102,"num_write":537,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1e","version":"141'829","reported_seq":3874,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.394718+0000","last_change":"2024-09-21T23:12:23.579643+0000","last_active":"2024-09-21T23:12:25.394718+0000","last_peered":"2024-09-21T23:12:25.394718+0000","last_clean":"2024-09-21T23:12:25.394718+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:12:25.394718+0000","last_undegraded":"2024-09-21T23:12:25.394718+0000","last_fullsized":"2024-09-21T23:12:25.394718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:38:50.812249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039144449999999999,"stat_sum":{"num_bytes":39916161,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":5425,"num_read_kb":41104,"num_write":1105,"num_write_kb":19714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1d","version":"144'412","reported_seq":3070,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.406573+0000","last_change":"2024-09-21T23:12:23.592278+0000","last_active":"2024-09-21T23:12:25.406573+0000","last_peered":"2024-09-21T23:12:25.406573+0000","last_clean":"2024-09-21T23:12:25.406573+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:12:25.406573+0000","last_undegraded":"2024-09-21T23:12:25.406573+0000","last_fullsized":"2024-09-21T23:12:25.406573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:52.418087+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.010589256999999999,"stat_sum":{"num_bytes":33006114,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":3058,"num_read_kb":13572,"num_write":352,"num_write_kb":11738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1c","version":"144'424","reported_seq":5330,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.519289+0000","last_change":"2024-09-21T23:12:23.583173+0000","last_active":"2024-09-21T23:12:25.519289+0000","last_peered":"2024-09-21T23:12:25.519289+0000","last_clean":"2024-09-21T23:12:25.519289+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:12:25.519289+0000","last_undegraded":"2024-09-21T23:12:25.519289+0000","last_fullsized":"2024-09-21T23:12:25.519289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:54:49.898758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001457943,"stat_sum":{"num_bytes":24933528,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":5026,"num_read_kb":22250,"num_write":448,"num_write_kb":12624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1b","version":"144'534","reported_seq":4011,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.557586+0000","last_change":"2024-09-21T23:12:23.591752+0000","last_active":"2024-09-21T23:12:25.557586+0000","last_peered":"2024-09-21T23:12:25.557586+0000","last_clean":"2024-09-21T23:12:25.557586+0000","last_became_active":"2024-09-21T23:11:07.389449+0000","last_became_peered":"2024-09-21T23:11:07.389449+0000","last_unstale":"2024-09-21T23:12:25.557586+0000","last_undegraded":"2024-09-21T23:12:25.557586+0000","last_fullsized":"2024-09-21T23:12:25.557586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:33.269776+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015889197000000001,"stat_sum":{"num_bytes":20680893,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":32,"num_read":4967,"num_read_kb":27575,"num_write":639,"num_write_kb":17922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1a","version":"140'1197","reported_seq":7075,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.500800+0000","last_change":"2024-09-21T23:12:25.474758+0000","last_active":"2024-09-21T23:12:25.500800+0000","last_peered":"2024-09-21T23:12:25.500800+0000","last_clean":"2024-09-21T23:12:22.791916+0000","last_became_active":"2024-09-21T23:12:25.472859+0000","last_became_peered":"2024-09-21T23:12:25.472859+0000","last_unstale":"2024-09-21T23:12:25.500800+0000","last_undegraded":"2024-09-21T23:12:25.396632+0000","last_fullsized":"2024-09-21T23:12:25.392722+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1197,"log_dups_size":0,"ondisk_log_size":1197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084651799999999999,"stat_sum":{"num_bytes":25351706,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":36,"num_read":11236,"num_read_kb":29005,"num_write":1596,"num_write_kb":9623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":7665879,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":65}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.19","version":"142'950","reported_seq":2851,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.391716+0000","last_change":"2024-09-21T23:12:23.587958+0000","last_active":"2024-09-21T23:12:25.391716+0000","last_peered":"2024-09-21T23:12:25.391716+0000","last_clean":"2024-09-21T23:12:25.391716+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:12:25.391716+0000","last_undegraded":"2024-09-21T23:12:25.391716+0000","last_fullsized":"2024-09-21T23:12:25.391716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:19:45.654621+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0075067440000000001,"stat_sum":{"num_bytes":18634995,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":31,"num_read":6236,"num_read_kb":21906,"num_write":1413,"num_write_kb":13868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.18","version":"144'1005","reported_seq":5232,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.547471+0000","last_change":"2024-09-21T23:12:23.596397+0000","last_active":"2024-09-21T23:12:25.547471+0000","last_peered":"2024-09-21T23:12:25.547471+0000","last_clean":"2024-09-21T23:12:25.547471+0000","last_became_active":"2024-09-21T23:10:52.807974+0000","last_became_peered":"2024-09-21T23:10:52.807974+0000","last_unstale":"2024-09-21T23:12:25.547471+0000","last_undegraded":"2024-09-21T23:12:25.547471+0000","last_fullsized":"2024-09-21T23:12:25.547471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:11:14.363064+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014775948000000001,"stat_sum":{"num_bytes":27400001,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":8064,"num_read_kb":28905,"num_write":1405,"num_write_kb":9717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3330203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.17","version":"142'605","reported_seq":5438,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400283+0000","last_change":"2024-09-21T23:12:23.595807+0000","last_active":"2024-09-21T23:12:25.400283+0000","last_peered":"2024-09-21T23:12:25.400283+0000","last_clean":"2024-09-21T23:12:25.400283+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:12:25.400283+0000","last_undegraded":"2024-09-21T23:12:25.400283+0000","last_fullsized":"2024-09-21T23:12:25.400283+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:19:05.865092+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018468425,"stat_sum":{"num_bytes":49521309,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":40,"num_read":6004,"num_read_kb":23039,"num_write":579,"num_write_kb":12193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.16","version":"140'470","reported_seq":4209,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.402173+0000","last_change":"2024-09-21T23:12:23.582175+0000","last_active":"2024-09-21T23:12:25.402173+0000","last_peered":"2024-09-21T23:12:25.402173+0000","last_clean":"2024-09-21T23:12:25.402173+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:12:25.402173+0000","last_undegraded":"2024-09-21T23:12:25.402173+0000","last_fullsized":"2024-09-21T23:12:25.402173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:50.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095586499999999997,"stat_sum":{"num_bytes":35841524,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":30,"num_read":5486,"num_read_kb":29495,"num_write":392,"num_write_kb":12526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.15","version":"142'460","reported_seq":6428,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.396907+0000","last_change":"2024-09-21T23:12:23.594752+0000","last_active":"2024-09-21T23:12:25.396907+0000","last_peered":"2024-09-21T23:12:25.396907+0000","last_clean":"2024-09-21T23:12:25.396907+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:12:25.396907+0000","last_undegraded":"2024-09-21T23:12:25.396907+0000","last_fullsized":"2024-09-21T23:12:25.396907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:58:42.593943+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015044037,"stat_sum":{"num_bytes":60921442,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":26,"num_read":6990,"num_read_kb":36195,"num_write":399,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.14","version":"136'648","reported_seq":4713,"reported_epoch":144,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-21T23:12:25.468450+0000","last_change":"2024-09-21T23:12:25.468450+0000","last_active":"2024-09-21T23:12:25.468450+0000","last_peered":"2024-09-21T23:12:25.468450+0000","last_clean":"2024-09-21T23:12:17.521065+0000","last_became_active":"2024-09-21T23:12:25.465572+0000","last_became_peered":"2024-09-21T23:12:25.465572+0000","last_unstale":"2024-09-21T23:12:25.468450+0000","last_undegraded":"2024-09-21T23:12:25.468450+0000","last_fullsized":"2024-09-21T23:12:25.468450+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002540123,"stat_sum":{"num_bytes":12681967,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":136,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":29,"num_read":5325,"num_read_kb":13911,"num_write":789,"num_write_kb":8285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":10037734,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":44}],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399605+0000","last_change":"2024-09-21T23:11:11.428751+0000","last_active":"2024-09-21T23:12:25.399605+0000","last_peered":"2024-09-21T23:12:25.399605+0000","last_clean":"2024-09-21T23:12:25.399605+0000","last_became_active":"2024-09-21T23:11:11.427806+0000","last_became_peered":"2024-09-21T23:11:11.427806+0000","last_unstale":"2024-09-21T23:12:25.399605+0000","last_undegraded":"2024-09-21T23:12:25.399605+0000","last_fullsized":"2024-09-21T23:12:25.399605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:40.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"144'567","reported_seq":2538,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398529+0000","last_change":"2024-09-21T23:12:23.595922+0000","last_active":"2024-09-21T23:12:25.398529+0000","last_peered":"2024-09-21T23:12:25.398529+0000","last_clean":"2024-09-21T23:12:25.398529+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:12:25.398529+0000","last_undegraded":"2024-09-21T23:12:25.398529+0000","last_fullsized":"2024-09-21T23:12:25.398529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:33:08.254017+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.021543285999999998,"stat_sum":{"num_bytes":51553571,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":32,"num_read":3092,"num_read_kb":25059,"num_write":596,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392423+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:12:25.392423+0000","last_peered":"2024-09-21T23:12:25.392423+0000","last_clean":"2024-09-21T23:12:25.392423+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:12:25.392423+0000","last_undegraded":"2024-09-21T23:12:25.392423+0000","last_fullsized":"2024-09-21T23:12:25.392423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"139'735","reported_seq":5701,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395364+0000","last_change":"2024-09-21T23:12:23.579231+0000","last_active":"2024-09-21T23:12:25.395364+0000","last_peered":"2024-09-21T23:12:25.395364+0000","last_clean":"2024-09-21T23:12:25.395364+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:12:25.395364+0000","last_undegraded":"2024-09-21T23:12:25.395364+0000","last_fullsized":"2024-09-21T23:12:25.395364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:15:27.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037482779999999999,"stat_sum":{"num_bytes":59845519,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":24,"num_read":7997,"num_read_kb":30135,"num_write":1011,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398221+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:12:25.398221+0000","last_peered":"2024-09-21T23:12:25.398221+0000","last_clean":"2024-09-21T23:12:25.398221+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:12:25.398221+0000","last_undegraded":"2024-09-21T23:12:25.398221+0000","last_fullsized":"2024-09-21T23:12:25.398221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"144'609","reported_seq":3657,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.548797+0000","last_change":"2024-09-21T23:12:23.591709+0000","last_active":"2024-09-21T23:12:25.548797+0000","last_peered":"2024-09-21T23:12:25.548797+0000","last_clean":"2024-09-21T23:12:25.548797+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:12:25.548797+0000","last_undegraded":"2024-09-21T23:12:25.548797+0000","last_fullsized":"2024-09-21T23:12:25.548797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:19.665608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097742330000000002,"stat_sum":{"num_bytes":23242291,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":5031,"num_read_kb":24018,"num_write":567,"num_write_kb":10680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392128+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:12:25.392128+0000","last_peered":"2024-09-21T23:12:25.392128+0000","last_clean":"2024-09-21T23:12:25.392128+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:12:25.392128+0000","last_undegraded":"2024-09-21T23:12:25.392128+0000","last_fullsized":"2024-09-21T23:12:25.392128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"141'761","reported_seq":5770,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.407237+0000","last_change":"2024-09-21T23:12:23.584716+0000","last_active":"2024-09-21T23:12:25.407237+0000","last_peered":"2024-09-21T23:12:25.407237+0000","last_clean":"2024-09-21T23:12:25.407237+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:12:25.407237+0000","last_undegraded":"2024-09-21T23:12:25.407237+0000","last_fullsized":"2024-09-21T23:12:25.407237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:27.235272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002790463,"stat_sum":{"num_bytes":60691725,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":33,"num_read":6756,"num_read_kb":41318,"num_write":1076,"num_write_kb":18307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.3","version":"142'660","reported_seq":4056,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.393364+0000","last_change":"2024-09-21T23:12:23.573763+0000","last_active":"2024-09-21T23:12:25.393364+0000","last_peered":"2024-09-21T23:12:25.393364+0000","last_clean":"2024-09-21T23:12:25.393364+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:12:25.393364+0000","last_undegraded":"2024-09-21T23:12:25.393364+0000","last_fullsized":"2024-09-21T23:12:25.393364+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:16:31.132834+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041488519999999997,"stat_sum":{"num_bytes":28607623,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":5511,"num_read_kb":26445,"num_write":659,"num_write_kb":12055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.2","version":"20'2","reported_seq":270,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400665+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:12:25.400665+0000","last_peered":"2024-09-21T23:12:25.400665+0000","last_clean":"2024-09-21T23:12:25.400665+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:12:25.400665+0000","last_undegraded":"2024-09-21T23:12:25.400665+0000","last_fullsized":"2024-09-21T23:12:25.400665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"142'532","reported_seq":4439,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.391720+0000","last_change":"2024-09-21T23:12:23.594569+0000","last_active":"2024-09-21T23:12:25.391720+0000","last_peered":"2024-09-21T23:12:25.391720+0000","last_clean":"2024-09-21T23:12:25.391720+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:12:25.391720+0000","last_undegraded":"2024-09-21T23:12:25.391720+0000","last_fullsized":"2024-09-21T23:12:25.391720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:03:19.335888+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0138751,"stat_sum":{"num_bytes":23945683,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":32,"num_read":6639,"num_read_kb":20514,"num_write":470,"num_write_kb":13136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399873+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:12:25.399873+0000","last_peered":"2024-09-21T23:12:25.399873+0000","last_clean":"2024-09-21T23:12:25.399873+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:12:25.399873+0000","last_undegraded":"2024-09-21T23:12:25.399873+0000","last_fullsized":"2024-09-21T23:12:25.399873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"144'439","reported_seq":5223,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.459920+0000","last_change":"2024-09-21T23:12:25.459920+0000","last_active":"2024-09-21T23:12:25.459920+0000","last_peered":"2024-09-21T23:12:25.459920+0000","last_clean":"2024-09-21T23:12:22.803009+0000","last_became_active":"2024-09-21T23:12:25.458543+0000","last_became_peered":"2024-09-21T23:12:25.458543+0000","last_unstale":"2024-09-21T23:12:25.459920+0000","last_undegraded":"2024-09-21T23:12:25.400767+0000","last_fullsized":"2024-09-21T23:12:25.398785+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074971900000000004,"stat_sum":{"num_bytes":29727255,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":30,"num_read":4756,"num_read_kb":20310,"num_write":488,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":6739606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,3","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":140,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395082+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:12:25.395082+0000","last_peered":"2024-09-21T23:12:25.395082+0000","last_clean":"2024-09-21T23:12:25.395082+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:12:25.395082+0000","last_undegraded":"2024-09-21T23:12:25.395082+0000","last_fullsized":"2024-09-21T23:12:25.395082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"141'541","reported_seq":4182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395824+0000","last_change":"2024-09-21T23:12:23.578066+0000","last_active":"2024-09-21T23:12:25.395824+0000","last_peered":"2024-09-21T23:12:25.395824+0000","last_clean":"2024-09-21T23:12:25.395824+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:12:25.395824+0000","last_undegraded":"2024-09-21T23:12:25.395824+0000","last_fullsized":"2024-09-21T23:12:25.395824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:23.184287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032415339999999999,"stat_sum":{"num_bytes":6317009,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":5243,"num_read_kb":19488,"num_write":626,"num_write_kb":14848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.3","version":"18'1","reported_seq":269,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400115+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:12:25.400115+0000","last_peered":"2024-09-21T23:12:25.400115+0000","last_clean":"2024-09-21T23:12:25.400115+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:12:25.400115+0000","last_undegraded":"2024-09-21T23:12:25.400115+0000","last_fullsized":"2024-09-21T23:12:25.400115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399214+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:12:25.399214+0000","last_peered":"2024-09-21T23:12:25.399214+0000","last_clean":"2024-09-21T23:12:25.399214+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:12:25.399214+0000","last_undegraded":"2024-09-21T23:12:25.399214+0000","last_fullsized":"2024-09-21T23:12:25.399214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"142'474","reported_seq":3347,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392920+0000","last_change":"2024-09-21T23:12:23.581501+0000","last_active":"2024-09-21T23:12:25.392920+0000","last_peered":"2024-09-21T23:12:25.392920+0000","last_clean":"2024-09-21T23:12:25.392920+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:12:25.392920+0000","last_undegraded":"2024-09-21T23:12:25.392920+0000","last_fullsized":"2024-09-21T23:12:25.392920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:23:33.737770+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0066331569999999998,"stat_sum":{"num_bytes":5864293,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":4360,"num_read_kb":9771,"num_write":470,"num_write_kb":5012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.9","version":"140'473","reported_seq":3988,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.457106+0000","last_change":"2024-09-21T23:12:25.457459+0000","last_active":"2024-09-21T23:12:25.457106+0000","last_peered":"2024-09-21T23:12:25.457106+0000","last_clean":"2024-09-21T23:12:22.773644+0000","last_became_active":"2024-09-21T23:12:25.456530+0000","last_became_peered":"2024-09-21T23:12:25.456530+0000","last_unstale":"2024-09-21T23:12:25.457106+0000","last_undegraded":"2024-09-21T23:12:25.399079+0000","last_fullsized":"2024-09-21T23:12:25.395974+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053112500000000002,"stat_sum":{"num_bytes":22697745,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":5637,"num_read_kb":19185,"num_write":443,"num_write_kb":5926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":4244532,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[2],"avail_no_missing":["2","7"],"object_location_counts":[{"shards":"2,7","objects":45}],"blocked_by":[],"up_primary":4,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.a","version":"140'568","reported_seq":5987,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.462692+0000","last_change":"2024-09-21T23:12:25.463123+0000","last_active":"2024-09-21T23:12:25.462692+0000","last_peered":"2024-09-21T23:12:25.462692+0000","last_clean":"2024-09-21T23:12:22.780649+0000","last_became_active":"2024-09-21T23:12:25.460731+0000","last_became_peered":"2024-09-21T23:12:25.460731+0000","last_unstale":"2024-09-21T23:12:25.462692+0000","last_undegraded":"2024-09-21T23:12:25.405385+0000","last_fullsized":"2024-09-21T23:12:25.401307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016923999999999999,"stat_sum":{"num_bytes":28480289,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":35,"num_read":6682,"num_read_kb":40112,"num_write":509,"num_write_kb":11929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":16052830,"num_keys_recovered":97,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":["6","1"],"object_location_counts":[{"shards":"1,6","objects":63}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.b","version":"138'353","reported_seq":3407,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.393434+0000","last_change":"2024-09-21T23:12:23.578585+0000","last_active":"2024-09-21T23:12:25.393434+0000","last_peered":"2024-09-21T23:12:25.393434+0000","last_clean":"2024-09-21T23:12:25.393434+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:12:25.393434+0000","last_undegraded":"2024-09-21T23:12:25.393434+0000","last_fullsized":"2024-09-21T23:12:25.393434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:27:01.335640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035367049999999998,"stat_sum":{"num_bytes":22020270,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":3313,"num_read_kb":26228,"num_write":320,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.c","version":"139'494","reported_seq":2095,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.401637+0000","last_change":"2024-09-21T23:12:23.582319+0000","last_active":"2024-09-21T23:12:25.401637+0000","last_peered":"2024-09-21T23:12:25.401637+0000","last_clean":"2024-09-21T23:12:25.401637+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:12:25.401637+0000","last_undegraded":"2024-09-21T23:12:25.401637+0000","last_fullsized":"2024-09-21T23:12:25.401637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:30:54.665289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011079219999999999,"stat_sum":{"num_bytes":19109519,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":38,"num_read":2206,"num_read_kb":17676,"num_write":471,"num_write_kb":9186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.d","version":"142'500","reported_seq":5026,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395926+0000","last_change":"2024-09-21T23:12:23.584209+0000","last_active":"2024-09-21T23:12:25.395926+0000","last_peered":"2024-09-21T23:12:25.395926+0000","last_clean":"2024-09-21T23:12:25.395926+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:12:25.395926+0000","last_undegraded":"2024-09-21T23:12:25.395926+0000","last_fullsized":"2024-09-21T23:12:25.395926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:52:18.266627+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070675219999999997,"stat_sum":{"num_bytes":53356632,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":38,"num_read":5085,"num_read_kb":28814,"num_write":425,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.e","version":"141'457","reported_seq":4530,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.467578+0000","last_change":"2024-09-21T23:12:25.468015+0000","last_active":"2024-09-21T23:12:25.467578+0000","last_peered":"2024-09-21T23:12:25.467578+0000","last_clean":"2024-09-21T23:12:22.697992+0000","last_became_active":"2024-09-21T23:12:25.465677+0000","last_became_peered":"2024-09-21T23:12:25.465677+0000","last_unstale":"2024-09-21T23:12:25.467578+0000","last_undegraded":"2024-09-21T23:12:25.400587+0000","last_fullsized":"2024-09-21T23:12:25.395088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028850960000000002,"stat_sum":{"num_bytes":47211785,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":35,"num_read":6688,"num_read_kb":23677,"num_write":456,"num_write_kb":9490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":16260526,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"1,7","objects":56}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.f","version":"144'537","reported_seq":2374,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.396129+0000","last_change":"2024-09-21T23:12:23.589358+0000","last_active":"2024-09-21T23:12:25.396129+0000","last_peered":"2024-09-21T23:12:25.396129+0000","last_clean":"2024-09-21T23:12:25.396129+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:12:25.396129+0000","last_undegraded":"2024-09-21T23:12:25.396129+0000","last_fullsized":"2024-09-21T23:12:25.396129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:01:53.460212+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010235556,"stat_sum":{"num_bytes":17603506,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":38,"num_read":2338,"num_read_kb":21657,"num_write":593,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.10","version":"142'610","reported_seq":3880,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.397340+0000","last_change":"2024-09-21T23:12:23.596504+0000","last_active":"2024-09-21T23:12:25.397340+0000","last_peered":"2024-09-21T23:12:25.397340+0000","last_clean":"2024-09-21T23:12:25.397340+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:12:25.397340+0000","last_undegraded":"2024-09-21T23:12:25.397340+0000","last_fullsized":"2024-09-21T23:12:25.397340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:49.322900+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013222350000000001,"stat_sum":{"num_bytes":50187387,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":5870,"num_read_kb":37397,"num_write":642,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.11","version":"142'616","reported_seq":6748,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395599+0000","last_change":"2024-09-21T23:12:23.588449+0000","last_active":"2024-09-21T23:12:25.395599+0000","last_peered":"2024-09-21T23:12:25.395599+0000","last_clean":"2024-09-21T23:12:25.395599+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:12:25.395599+0000","last_undegraded":"2024-09-21T23:12:25.395599+0000","last_fullsized":"2024-09-21T23:12:25.395599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:24.702332+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.011401987000000001,"stat_sum":{"num_bytes":60771037,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":7995,"num_read_kb":42900,"num_write":653,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.12","version":"142'546","reported_seq":6812,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398361+0000","last_change":"2024-09-21T23:12:23.578594+0000","last_active":"2024-09-21T23:12:25.398361+0000","last_peered":"2024-09-21T23:12:25.398361+0000","last_clean":"2024-09-21T23:12:25.398361+0000","last_became_active":"2024-09-21T23:11:15.382800+0000","last_became_peered":"2024-09-21T23:11:15.382800+0000","last_unstale":"2024-09-21T23:12:25.398361+0000","last_undegraded":"2024-09-21T23:12:25.398361+0000","last_fullsized":"2024-09-21T23:12:25.398361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:55.305903+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0044734670000000001,"stat_sum":{"num_bytes":21979199,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":7663,"num_read_kb":39292,"num_write":614,"num_write_kb":21738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.13","version":"141'384","reported_seq":2162,"reported_epoch":144,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-21T23:12:25.551458+0000","last_change":"2024-09-21T23:12:25.465200+0000","last_active":"2024-09-21T23:12:25.551458+0000","last_peered":"2024-09-21T23:12:25.551458+0000","last_clean":"2024-09-21T23:12:22.713475+0000","last_became_active":"2024-09-21T23:12:25.446879+0000","last_became_peered":"2024-09-21T23:12:25.446879+0000","last_unstale":"2024-09-21T23:12:25.551458+0000","last_undegraded":"2024-09-21T23:12:25.551458+0000","last_fullsized":"2024-09-21T23:12:25.395801+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440593,"stat_sum":{"num_bytes":17303039,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":33,"num_read":1454,"num_read_kb":14144,"num_write":371,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":19217578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","2"],"object_location_counts":[{"shards":"2,7","objects":29},{"shards":"7","objects":22}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1e","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1024360599,"num_objects":1646,"num_object_clones":294,"num_object_copies":3292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":1646,"num_whiteouts":963,"num_read":178463,"num_read_kb":834199,"num_write":21114,"num_write_kb":399614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3564,"num_bytes_recovered":226891202,"num_keys_recovered":600,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401350656,"data_stored":426810805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":18947,"ondisk_log_size":18947,"up":64,"acting":58,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124892,"kb_used_data":71496,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94246948,"statfs":{"total":96636764160,"available":96508874752,"internally_reserved":0,"allocated":73211904,"data_stored":74382176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70260,"kb_used_data":30744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94301580,"statfs":{"total":96636764160,"available":96564817920,"internally_reserved":0,"allocated":31481856,"data_stored":31064524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":50644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":51859456,"data_stored":58668357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743442,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66224,"kb_used_data":37592,"kb_used_omap":15,"kb_used_meta":28592,"kb_avail":94305616,"statfs":{"total":96636764160,"available":96568950784,"internally_reserved":0,"allocated":38494208,"data_stored":40217465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":29279170},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140912,"kb_used_data":69468,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94230928,"statfs":{"total":96636764160,"available":96492470272,"internally_reserved":0,"allocated":71135232,"data_stored":73991323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68432,"kb_used_data":28920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94303408,"statfs":{"total":96636764160,"available":96566689792,"internally_reserved":0,"allocated":29614080,"data_stored":30458988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97236,"kb_used_data":48000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94274604,"statfs":{"total":96636764160,"available":96537194496,"internally_reserved":0,"allocated":49152000,"data_stored":50324918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123260,"kb_used_data":69860,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94248580,"statfs":{"total":96636764160,"available":96510545920,"internally_reserved":0,"allocated":71536640,"data_stored":75778511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69091328,"data_stored":74163552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":47431680,"data_stored":49496954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27885568,"data_stored":29624898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69419008,"data_stored":73163378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36917248,"data_stored":39517697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50020352,"data_stored":57715581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29765632,"data_stored":30236579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70819840,"data_stored":72892166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:12:27.040 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:12:27.113 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:27.109+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:27.199 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:27.199 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:27.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.199 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123547 3.13 3.8d034793 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.45706713700000001, 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.45710763599999998, 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for peered", 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123547 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742119+0000", 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0244e-05 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742127+0000", 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6100000000000006e-06 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742154+0000", 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6730999999999999e-05 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742165+0000", 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1419e-05 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742241+0000", 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5562999999999995e-05 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742251+0000", 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0017000000000001e-05 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:12:27.203 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:27.214 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:27.209+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:27.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:27.315 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:27.312+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:27.341 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:27.341 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:27.349 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:12:27.350 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:12:27.359 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:27.368 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-21T23:12:26.608678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1024950986,"num_objects":1650,"num_object_clones":294,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":963,"num_read":178579,"num_read_kb":834299,"num_write":21275,"num_write_kb":402032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3568,"num_bytes_recovered":228071938,"num_keys_recovered":600,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19067,"ondisk_log_size":19067,"up":82,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":768888,"kb_used_data":406724,"kb_used_omap":25,"kb_used_meta":361958,"kb_avail":754205832,"statfs":{"total":773094113280,"available":772306771968,"internally_reserved":0,"allocated":416485376,"data_stored":434886262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":370645039},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-107280653,"num_objects":-84,"num_object_clones":-34,"num_object_copies":-168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":-84,"num_whiteouts":-19,"num_read":721,"num_read_kb":616,"num_write":122,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1199468,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.041663"},"pg_stats":[{"pgid":"3.1f","version":"136'557","reported_seq":3959,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400095+0000","last_change":"2024-09-21T23:12:23.575904+0000","last_active":"2024-09-21T23:12:25.400095+0000","last_peered":"2024-09-21T23:12:25.400095+0000","last_clean":"2024-09-21T23:12:25.400095+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:12:25.400095+0000","last_undegraded":"2024-09-21T23:12:25.400095+0000","last_fullsized":"2024-09-21T23:12:25.400095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:06:35.248845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017863670000000001,"stat_sum":{"num_bytes":24955582,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":30,"num_read":6330,"num_read_kb":19102,"num_write":537,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1e","version":"141'829","reported_seq":3874,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.394718+0000","last_change":"2024-09-21T23:12:23.579643+0000","last_active":"2024-09-21T23:12:25.394718+0000","last_peered":"2024-09-21T23:12:25.394718+0000","last_clean":"2024-09-21T23:12:25.394718+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:12:25.394718+0000","last_undegraded":"2024-09-21T23:12:25.394718+0000","last_fullsized":"2024-09-21T23:12:25.394718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:38:50.812249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039144449999999999,"stat_sum":{"num_bytes":39916161,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":5425,"num_read_kb":41104,"num_write":1105,"num_write_kb":19714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1d","version":"144'412","reported_seq":3070,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.406573+0000","last_change":"2024-09-21T23:12:23.592278+0000","last_active":"2024-09-21T23:12:25.406573+0000","last_peered":"2024-09-21T23:12:25.406573+0000","last_clean":"2024-09-21T23:12:25.406573+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:12:25.406573+0000","last_undegraded":"2024-09-21T23:12:25.406573+0000","last_fullsized":"2024-09-21T23:12:25.406573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:52.418087+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.010589256999999999,"stat_sum":{"num_bytes":33006114,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":3058,"num_read_kb":13572,"num_write":352,"num_write_kb":11738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1c","version":"144'424","reported_seq":5330,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.519289+0000","last_change":"2024-09-21T23:12:23.583173+0000","last_active":"2024-09-21T23:12:25.519289+0000","last_peered":"2024-09-21T23:12:25.519289+0000","last_clean":"2024-09-21T23:12:25.519289+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:12:25.519289+0000","last_undegraded":"2024-09-21T23:12:25.519289+0000","last_fullsized":"2024-09-21T23:12:25.519289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:54:49.898758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001457943,"stat_sum":{"num_bytes":24933528,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":5026,"num_read_kb":22250,"num_write":448,"num_write_kb":12624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1b","version":"144'534","reported_seq":4011,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.557586+0000","last_change":"2024-09-21T23:12:23.591752+0000","last_active":"2024-09-21T23:12:25.557586+0000","last_peered":"2024-09-21T23:12:25.557586+0000","last_clean":"2024-09-21T23:12:25.557586+0000","last_became_active":"2024-09-21T23:11:07.389449+0000","last_became_peered":"2024-09-21T23:11:07.389449+0000","last_unstale":"2024-09-21T23:12:25.557586+0000","last_undegraded":"2024-09-21T23:12:25.557586+0000","last_fullsized":"2024-09-21T23:12:25.557586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:33.269776+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015889197000000001,"stat_sum":{"num_bytes":20680893,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":32,"num_read":4967,"num_read_kb":27575,"num_write":639,"num_write_kb":17922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1a","version":"140'1197","reported_seq":7075,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.500800+0000","last_change":"2024-09-21T23:12:25.474758+0000","last_active":"2024-09-21T23:12:25.500800+0000","last_peered":"2024-09-21T23:12:25.500800+0000","last_clean":"2024-09-21T23:12:22.791916+0000","last_became_active":"2024-09-21T23:12:25.472859+0000","last_became_peered":"2024-09-21T23:12:25.472859+0000","last_unstale":"2024-09-21T23:12:25.500800+0000","last_undegraded":"2024-09-21T23:12:25.396632+0000","last_fullsized":"2024-09-21T23:12:25.392722+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1197,"log_dups_size":0,"ondisk_log_size":1197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084651799999999999,"stat_sum":{"num_bytes":25351706,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":36,"num_read":11236,"num_read_kb":29005,"num_write":1596,"num_write_kb":9623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":7665879,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":65}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.19","version":"142'950","reported_seq":2851,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.391716+0000","last_change":"2024-09-21T23:12:23.587958+0000","last_active":"2024-09-21T23:12:25.391716+0000","last_peered":"2024-09-21T23:12:25.391716+0000","last_clean":"2024-09-21T23:12:25.391716+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:12:25.391716+0000","last_undegraded":"2024-09-21T23:12:25.391716+0000","last_fullsized":"2024-09-21T23:12:25.391716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:19:45.654621+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0075067440000000001,"stat_sum":{"num_bytes":18634995,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":31,"num_read":6236,"num_read_kb":21906,"num_write":1413,"num_write_kb":13868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.18","version":"144'1005","reported_seq":5232,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.547471+0000","last_change":"2024-09-21T23:12:23.596397+0000","last_active":"2024-09-21T23:12:25.547471+0000","last_peered":"2024-09-21T23:12:25.547471+0000","last_clean":"2024-09-21T23:12:25.547471+0000","last_became_active":"2024-09-21T23:10:52.807974+0000","last_became_peered":"2024-09-21T23:10:52.807974+0000","last_unstale":"2024-09-21T23:12:25.547471+0000","last_undegraded":"2024-09-21T23:12:25.547471+0000","last_fullsized":"2024-09-21T23:12:25.547471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:11:14.363064+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014775948000000001,"stat_sum":{"num_bytes":27400001,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":8064,"num_read_kb":28905,"num_write":1405,"num_write_kb":9717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3330203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.17","version":"142'605","reported_seq":5438,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400283+0000","last_change":"2024-09-21T23:12:23.595807+0000","last_active":"2024-09-21T23:12:25.400283+0000","last_peered":"2024-09-21T23:12:25.400283+0000","last_clean":"2024-09-21T23:12:25.400283+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:12:25.400283+0000","last_undegraded":"2024-09-21T23:12:25.400283+0000","last_fullsized":"2024-09-21T23:12:25.400283+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:19:05.865092+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018468425,"stat_sum":{"num_bytes":49521309,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":40,"num_read":6004,"num_read_kb":23039,"num_write":579,"num_write_kb":12193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.16","version":"140'470","reported_seq":4209,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.402173+0000","last_change":"2024-09-21T23:12:23.582175+0000","last_active":"2024-09-21T23:12:25.402173+0000","last_peered":"2024-09-21T23:12:25.402173+0000","last_clean":"2024-09-21T23:12:25.402173+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:12:25.402173+0000","last_undegraded":"2024-09-21T23:12:25.402173+0000","last_fullsized":"2024-09-21T23:12:25.402173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:50.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095586499999999997,"stat_sum":{"num_bytes":35841524,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":30,"num_read":5486,"num_read_kb":29495,"num_write":392,"num_write_kb":12526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.15","version":"142'460","reported_seq":6428,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.396907+0000","last_change":"2024-09-21T23:12:23.594752+0000","last_active":"2024-09-21T23:12:25.396907+0000","last_peered":"2024-09-21T23:12:25.396907+0000","last_clean":"2024-09-21T23:12:25.396907+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:12:25.396907+0000","last_undegraded":"2024-09-21T23:12:25.396907+0000","last_fullsized":"2024-09-21T23:12:25.396907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:58:42.593943+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015044037,"stat_sum":{"num_bytes":60921442,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":26,"num_read":6990,"num_read_kb":36195,"num_write":399,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.14","version":"136'648","reported_seq":4713,"reported_epoch":144,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-21T23:12:25.468450+0000","last_change":"2024-09-21T23:12:25.468450+0000","last_active":"2024-09-21T23:12:25.468450+0000","last_peered":"2024-09-21T23:12:25.468450+0000","last_clean":"2024-09-21T23:12:17.521065+0000","last_became_active":"2024-09-21T23:12:25.465572+0000","last_became_peered":"2024-09-21T23:12:25.465572+0000","last_unstale":"2024-09-21T23:12:25.468450+0000","last_undegraded":"2024-09-21T23:12:25.468450+0000","last_fullsized":"2024-09-21T23:12:25.468450+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002540123,"stat_sum":{"num_bytes":12681967,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":136,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":29,"num_read":5325,"num_read_kb":13911,"num_write":789,"num_write_kb":8285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":10037734,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":44}],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399605+0000","last_change":"2024-09-21T23:11:11.428751+0000","last_active":"2024-09-21T23:12:25.399605+0000","last_peered":"2024-09-21T23:12:25.399605+0000","last_clean":"2024-09-21T23:12:25.399605+0000","last_became_active":"2024-09-21T23:11:11.427806+0000","last_became_peered":"2024-09-21T23:11:11.427806+0000","last_unstale":"2024-09-21T23:12:25.399605+0000","last_undegraded":"2024-09-21T23:12:25.399605+0000","last_fullsized":"2024-09-21T23:12:25.399605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:40.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"144'567","reported_seq":2538,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398529+0000","last_change":"2024-09-21T23:12:23.595922+0000","last_active":"2024-09-21T23:12:25.398529+0000","last_peered":"2024-09-21T23:12:25.398529+0000","last_clean":"2024-09-21T23:12:25.398529+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:12:25.398529+0000","last_undegraded":"2024-09-21T23:12:25.398529+0000","last_fullsized":"2024-09-21T23:12:25.398529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:33:08.254017+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.021543285999999998,"stat_sum":{"num_bytes":51553571,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":32,"num_read":3092,"num_read_kb":25059,"num_write":596,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392423+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:12:25.392423+0000","last_peered":"2024-09-21T23:12:25.392423+0000","last_clean":"2024-09-21T23:12:25.392423+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:12:25.392423+0000","last_undegraded":"2024-09-21T23:12:25.392423+0000","last_fullsized":"2024-09-21T23:12:25.392423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"139'735","reported_seq":5701,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395364+0000","last_change":"2024-09-21T23:12:23.579231+0000","last_active":"2024-09-21T23:12:25.395364+0000","last_peered":"2024-09-21T23:12:25.395364+0000","last_clean":"2024-09-21T23:12:25.395364+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:12:25.395364+0000","last_undegraded":"2024-09-21T23:12:25.395364+0000","last_fullsized":"2024-09-21T23:12:25.395364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:15:27.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037482779999999999,"stat_sum":{"num_bytes":59845519,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":24,"num_read":7997,"num_read_kb":30135,"num_write":1011,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398221+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:12:25.398221+0000","last_peered":"2024-09-21T23:12:25.398221+0000","last_clean":"2024-09-21T23:12:25.398221+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:12:25.398221+0000","last_undegraded":"2024-09-21T23:12:25.398221+0000","last_fullsized":"2024-09-21T23:12:25.398221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"144'609","reported_seq":3657,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.548797+0000","last_change":"2024-09-21T23:12:23.591709+0000","last_active":"2024-09-21T23:12:25.548797+0000","last_peered":"2024-09-21T23:12:25.548797+0000","last_clean":"2024-09-21T23:12:25.548797+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:12:25.548797+0000","last_undegraded":"2024-09-21T23:12:25.548797+0000","last_fullsized":"2024-09-21T23:12:25.548797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:19.665608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097742330000000002,"stat_sum":{"num_bytes":23242291,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":5031,"num_read_kb":24018,"num_write":567,"num_write_kb":10680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392128+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:12:25.392128+0000","last_peered":"2024-09-21T23:12:25.392128+0000","last_clean":"2024-09-21T23:12:25.392128+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:12:25.392128+0000","last_undegraded":"2024-09-21T23:12:25.392128+0000","last_fullsized":"2024-09-21T23:12:25.392128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"141'761","reported_seq":5770,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.407237+0000","last_change":"2024-09-21T23:12:23.584716+0000","last_active":"2024-09-21T23:12:25.407237+0000","last_peered":"2024-09-21T23:12:25.407237+0000","last_clean":"2024-09-21T23:12:25.407237+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:12:25.407237+0000","last_undegraded":"2024-09-21T23:12:25.407237+0000","last_fullsized":"2024-09-21T23:12:25.407237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:27.235272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002790463,"stat_sum":{"num_bytes":60691725,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":33,"num_read":6756,"num_read_kb":41318,"num_write":1076,"num_write_kb":18307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.3","version":"142'660","reported_seq":4056,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.393364+0000","last_change":"2024-09-21T23:12:23.573763+0000","last_active":"2024-09-21T23:12:25.393364+0000","last_peered":"2024-09-21T23:12:25.393364+0000","last_clean":"2024-09-21T23:12:25.393364+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:12:25.393364+0000","last_undegraded":"2024-09-21T23:12:25.393364+0000","last_fullsized":"2024-09-21T23:12:25.393364+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:16:31.132834+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041488519999999997,"stat_sum":{"num_bytes":28607623,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":5511,"num_read_kb":26445,"num_write":659,"num_write_kb":12055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.2","version":"20'2","reported_seq":270,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400665+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:12:25.400665+0000","last_peered":"2024-09-21T23:12:25.400665+0000","last_clean":"2024-09-21T23:12:25.400665+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:12:25.400665+0000","last_undegraded":"2024-09-21T23:12:25.400665+0000","last_fullsized":"2024-09-21T23:12:25.400665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"142'532","reported_seq":4439,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.391720+0000","last_change":"2024-09-21T23:12:23.594569+0000","last_active":"2024-09-21T23:12:25.391720+0000","last_peered":"2024-09-21T23:12:25.391720+0000","last_clean":"2024-09-21T23:12:25.391720+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:12:25.391720+0000","last_undegraded":"2024-09-21T23:12:25.391720+0000","last_fullsized":"2024-09-21T23:12:25.391720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:03:19.335888+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0138751,"stat_sum":{"num_bytes":23945683,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":32,"num_read":6639,"num_read_kb":20514,"num_write":470,"num_write_kb":13136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399873+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:12:25.399873+0000","last_peered":"2024-09-21T23:12:25.399873+0000","last_clean":"2024-09-21T23:12:25.399873+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:12:25.399873+0000","last_undegraded":"2024-09-21T23:12:25.399873+0000","last_fullsized":"2024-09-21T23:12:25.399873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"144'439","reported_seq":5223,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.459920+0000","last_change":"2024-09-21T23:12:25.459920+0000","last_active":"2024-09-21T23:12:25.459920+0000","last_peered":"2024-09-21T23:12:25.459920+0000","last_clean":"2024-09-21T23:12:22.803009+0000","last_became_active":"2024-09-21T23:12:25.458543+0000","last_became_peered":"2024-09-21T23:12:25.458543+0000","last_unstale":"2024-09-21T23:12:25.459920+0000","last_undegraded":"2024-09-21T23:12:25.400767+0000","last_fullsized":"2024-09-21T23:12:25.398785+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074971900000000004,"stat_sum":{"num_bytes":29727255,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":30,"num_read":4756,"num_read_kb":20310,"num_write":488,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":6739606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,3","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":140,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395082+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:12:25.395082+0000","last_peered":"2024-09-21T23:12:25.395082+0000","last_clean":"2024-09-21T23:12:25.395082+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:12:25.395082+0000","last_undegraded":"2024-09-21T23:12:25.395082+0000","last_fullsized":"2024-09-21T23:12:25.395082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"141'541","reported_seq":4182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395824+0000","last_change":"2024-09-21T23:12:23.578066+0000","last_active":"2024-09-21T23:12:25.395824+0000","last_peered":"2024-09-21T23:12:25.395824+0000","last_clean":"2024-09-21T23:12:25.395824+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:12:25.395824+0000","last_undegraded":"2024-09-21T23:12:25.395824+0000","last_fullsized":"2024-09-21T23:12:25.395824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:23.184287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032415339999999999,"stat_sum":{"num_bytes":6317009,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":5243,"num_read_kb":19488,"num_write":626,"num_write_kb":14848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.3","version":"18'1","reported_seq":269,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400115+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:12:25.400115+0000","last_peered":"2024-09-21T23:12:25.400115+0000","last_clean":"2024-09-21T23:12:25.400115+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:12:25.400115+0000","last_undegraded":"2024-09-21T23:12:25.400115+0000","last_fullsized":"2024-09-21T23:12:25.400115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399214+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:12:25.399214+0000","last_peered":"2024-09-21T23:12:25.399214+0000","last_clean":"2024-09-21T23:12:25.399214+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:12:25.399214+0000","last_undegraded":"2024-09-21T23:12:25.399214+0000","last_fullsized":"2024-09-21T23:12:25.399214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"142'474","reported_seq":3347,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392920+0000","last_change":"2024-09-21T23:12:23.581501+0000","last_active":"2024-09-21T23:12:25.392920+0000","last_peered":"2024-09-21T23:12:25.392920+0000","last_clean":"2024-09-21T23:12:25.392920+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:12:25.392920+0000","last_undegraded":"2024-09-21T23:12:25.392920+0000","last_fullsized":"2024-09-21T23:12:25.392920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:23:33.737770+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0066331569999999998,"stat_sum":{"num_bytes":5864293,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":4360,"num_read_kb":9771,"num_write":470,"num_write_kb":5012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.9","version":"140'473","reported_seq":3988,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.457106+0000","last_change":"2024-09-21T23:12:25.457459+0000","last_active":"2024-09-21T23:12:25.457106+0000","last_peered":"2024-09-21T23:12:25.457106+0000","last_clean":"2024-09-21T23:12:22.773644+0000","last_became_active":"2024-09-21T23:12:25.456530+0000","last_became_peered":"2024-09-21T23:12:25.456530+0000","last_unstale":"2024-09-21T23:12:25.457106+0000","last_undegraded":"2024-09-21T23:12:25.399079+0000","last_fullsized":"2024-09-21T23:12:25.395974+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053112500000000002,"stat_sum":{"num_bytes":22697745,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":5637,"num_read_kb":19185,"num_write":443,"num_write_kb":5926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":4244532,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[2],"avail_no_missing":["2","7"],"object_location_counts":[{"shards":"2,7","objects":45}],"blocked_by":[],"up_primary":4,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.a","version":"140'568","reported_seq":5987,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.462692+0000","last_change":"2024-09-21T23:12:25.463123+0000","last_active":"2024-09-21T23:12:25.462692+0000","last_peered":"2024-09-21T23:12:25.462692+0000","last_clean":"2024-09-21T23:12:22.780649+0000","last_became_active":"2024-09-21T23:12:25.460731+0000","last_became_peered":"2024-09-21T23:12:25.460731+0000","last_unstale":"2024-09-21T23:12:25.462692+0000","last_undegraded":"2024-09-21T23:12:25.405385+0000","last_fullsized":"2024-09-21T23:12:25.401307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016923999999999999,"stat_sum":{"num_bytes":28480289,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":35,"num_read":6682,"num_read_kb":40112,"num_write":509,"num_write_kb":11929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":16052830,"num_keys_recovered":97,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":["6","1"],"object_location_counts":[{"shards":"1,6","objects":63}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.b","version":"138'353","reported_seq":3407,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.393434+0000","last_change":"2024-09-21T23:12:23.578585+0000","last_active":"2024-09-21T23:12:25.393434+0000","last_peered":"2024-09-21T23:12:25.393434+0000","last_clean":"2024-09-21T23:12:25.393434+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:12:25.393434+0000","last_undegraded":"2024-09-21T23:12:25.393434+0000","last_fullsized":"2024-09-21T23:12:25.393434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:27:01.335640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035367049999999998,"stat_sum":{"num_bytes":22020270,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":3313,"num_read_kb":26228,"num_write":320,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.c","version":"139'494","reported_seq":2095,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.401637+0000","last_change":"2024-09-21T23:12:23.582319+0000","last_active":"2024-09-21T23:12:25.401637+0000","last_peered":"2024-09-21T23:12:25.401637+0000","last_clean":"2024-09-21T23:12:25.401637+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:12:25.401637+0000","last_undegraded":"2024-09-21T23:12:25.401637+0000","last_fullsized":"2024-09-21T23:12:25.401637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:30:54.665289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011079219999999999,"stat_sum":{"num_bytes":19109519,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":38,"num_read":2206,"num_read_kb":17676,"num_write":471,"num_write_kb":9186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.d","version":"142'500","reported_seq":5026,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395926+0000","last_change":"2024-09-21T23:12:23.584209+0000","last_active":"2024-09-21T23:12:25.395926+0000","last_peered":"2024-09-21T23:12:25.395926+0000","last_clean":"2024-09-21T23:12:25.395926+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:12:25.395926+0000","last_undegraded":"2024-09-21T23:12:25.395926+0000","last_fullsized":"2024-09-21T23:12:25.395926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:52:18.266627+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070675219999999997,"stat_sum":{"num_bytes":53356632,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":38,"num_read":5085,"num_read_kb":28814,"num_write":425,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.e","version":"141'457","reported_seq":4530,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.467578+0000","last_change":"2024-09-21T23:12:25.468015+0000","last_active":"2024-09-21T23:12:25.467578+0000","last_peered":"2024-09-21T23:12:25.467578+0000","last_clean":"2024-09-21T23:12:22.697992+0000","last_became_active":"2024-09-21T23:12:25.465677+0000","last_became_peered":"2024-09-21T23:12:25.465677+0000","last_unstale":"2024-09-21T23:12:25.467578+0000","last_undegraded":"2024-09-21T23:12:25.400587+0000","last_fullsized":"2024-09-21T23:12:25.395088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028850960000000002,"stat_sum":{"num_bytes":47211785,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":35,"num_read":6688,"num_read_kb":23677,"num_write":456,"num_write_kb":9490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":16260526,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"1,7","objects":56}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.f","version":"144'537","reported_seq":2374,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.396129+0000","last_change":"2024-09-21T23:12:23.589358+0000","last_active":"2024-09-21T23:12:25.396129+0000","last_peered":"2024-09-21T23:12:25.396129+0000","last_clean":"2024-09-21T23:12:25.396129+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:12:25.396129+0000","last_undegraded":"2024-09-21T23:12:25.396129+0000","last_fullsized":"2024-09-21T23:12:25.396129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:01:53.460212+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010235556,"stat_sum":{"num_bytes":17603506,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":38,"num_read":2338,"num_read_kb":21657,"num_write":593,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.10","version":"142'610","reported_seq":3880,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.397340+0000","last_change":"2024-09-21T23:12:23.596504+0000","last_active":"2024-09-21T23:12:25.397340+0000","last_peered":"2024-09-21T23:12:25.397340+0000","last_clean":"2024-09-21T23:12:25.397340+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:12:25.397340+0000","last_undegraded":"2024-09-21T23:12:25.397340+0000","last_fullsized":"2024-09-21T23:12:25.397340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:49.322900+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013222350000000001,"stat_sum":{"num_bytes":50187387,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":5870,"num_read_kb":37397,"num_write":642,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.11","version":"142'616","reported_seq":6748,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395599+0000","last_change":"2024-09-21T23:12:23.588449+0000","last_active":"2024-09-21T23:12:25.395599+0000","last_peered":"2024-09-21T23:12:25.395599+0000","last_clean":"2024-09-21T23:12:25.395599+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:12:25.395599+0000","last_undegraded":"2024-09-21T23:12:25.395599+0000","last_fullsized":"2024-09-21T23:12:25.395599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:24.702332+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.011401987000000001,"stat_sum":{"num_bytes":60771037,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":7995,"num_read_kb":42900,"num_write":653,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.12","version":"142'546","reported_seq":6812,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398361+0000","last_change":"2024-09-21T23:12:23.578594+0000","last_active":"2024-09-21T23:12:25.398361+0000","last_peered":"2024-09-21T23:12:25.398361+0000","last_clean":"2024-09-21T23:12:25.398361+0000","last_became_active":"2024-09-21T23:11:15.382800+0000","last_became_peered":"2024-09-21T23:11:15.382800+0000","last_unstale":"2024-09-21T23:12:25.398361+0000","last_undegraded":"2024-09-21T23:12:25.398361+0000","last_fullsized":"2024-09-21T23:12:25.398361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:55.305903+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0044734670000000001,"stat_sum":{"num_bytes":21979199,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":7663,"num_read_kb":39292,"num_write":614,"num_write_kb":21738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.13","version":"141'384","reported_seq":2162,"reported_epoch":144,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-21T23:12:25.551458+0000","last_change":"2024-09-21T23:12:25.465200+0000","last_active":"2024-09-21T23:12:25.551458+0000","last_peered":"2024-09-21T23:12:25.551458+0000","last_clean":"2024-09-21T23:12:22.713475+0000","last_became_active":"2024-09-21T23:12:25.446879+0000","last_became_peered":"2024-09-21T23:12:25.446879+0000","last_unstale":"2024-09-21T23:12:25.551458+0000","last_undegraded":"2024-09-21T23:12:25.551458+0000","last_fullsized":"2024-09-21T23:12:25.395801+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440593,"stat_sum":{"num_bytes":17303039,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":33,"num_read":1454,"num_read_kb":14144,"num_write":371,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":19217578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","2"],"object_location_counts":[{"shards":"2,7","objects":29},{"shards":"7","objects":22}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1e","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1024360599,"num_objects":1646,"num_object_clones":294,"num_object_copies":3292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":1646,"num_whiteouts":963,"num_read":178463,"num_read_kb":834199,"num_write":21114,"num_write_kb":399614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3564,"num_bytes_recovered":226891202,"num_keys_recovered":600,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401350656,"data_stored":426810805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":18947,"ondisk_log_size":18947,"up":64,"acting":58,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124892,"kb_used_data":71496,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94246948,"statfs":{"total":96636764160,"available":96508874752,"internally_reserved":0,"allocated":73211904,"data_stored":74382176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70260,"kb_used_data":30744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94301580,"statfs":{"total":96636764160,"available":96564817920,"internally_reserved":0,"allocated":31481856,"data_stored":31064524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":50644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":51859456,"data_stored":58668357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743442,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66224,"kb_used_data":37592,"kb_used_omap":15,"kb_used_meta":28592,"kb_avail":94305616,"statfs":{"total":96636764160,"available":96568950784,"internally_reserved":0,"allocated":38494208,"data_stored":40217465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":29279170},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140912,"kb_used_data":69468,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94230928,"statfs":{"total":96636764160,"available":96492470272,"internally_reserved":0,"allocated":71135232,"data_stored":73991323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68432,"kb_used_data":28920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94303408,"statfs":{"total":96636764160,"available":96566689792,"internally_reserved":0,"allocated":29614080,"data_stored":30458988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97236,"kb_used_data":48000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94274604,"statfs":{"total":96636764160,"available":96537194496,"internally_reserved":0,"allocated":49152000,"data_stored":50324918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123260,"kb_used_data":69860,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94248580,"statfs":{"total":96636764160,"available":96510545920,"internally_reserved":0,"allocated":71536640,"data_stored":75778511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69091328,"data_stored":74163552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":47431680,"data_stored":49496954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27885568,"data_stored":29624898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69419008,"data_stored":73163378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36917248,"data_stored":39517697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50020352,"data_stored":57715581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29765632,"data_stored":30236579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70819840,"data_stored":72892166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:12:27.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T23:12:27.457 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:27.453+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:27.518 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:27.513+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:27.553 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.366013608, 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.554 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:12:27.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:12:27.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.36591052200001, 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:12:27.558 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:12:27.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:27.562 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.365843173, 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:12:27.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:12:27.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.29065623700001, 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.566 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:27.570 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.290600469, 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:12:27.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:12:27.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.29056893000001, 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:12:27.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:12:27.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.29053618099999, 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:12:27.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:27.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:12:27.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:12:27.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.29050928800001, 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.583 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:12:27.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:12:27.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:12:27.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.290484015, 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.587 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:12:27.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:12:27.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.290452888, 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:12:27.591 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.595 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28980953199999, 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:27.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:12:27.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5945 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28462360500001, 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053199050999999997, 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5945 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265184+0000", 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265185+0000", 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:27.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265187+0000", 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265195+0000", 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1300000000000003e-06 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265199+0000", 2024-09-21T23:12:27.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5329999999999996e-06 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318154+0000", 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052954586999999997 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318253+0000", 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-05 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318383+0000", 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013029799999999999 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.284602122, 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:12:27.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28457783600001, 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.284560169, 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.612 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:12:27.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28453658399999, 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.616 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:27.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:12:27.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28452056899999, 2024-09-21T23:12:27.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:12:27.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:12:27.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.624 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.284506738, 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.628 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28447537700001, 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.629 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:12:27.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:12:27.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28440695500001, 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:27.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:12:27.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:12:27.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.637 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:27.639 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:12:27.639 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:27.639 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:27.636+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:27.739 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:27.733+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:27.789 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:12:27.789 INFO:teuthology.orchestra.run.smithi150.stdout:{"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 696/3300 objects degraded (21.091%), 5 pgs degraded","count":5},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":182,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":146,"num_osds":8,"num_up_osds":8,"osd_up_since":1726960261,"num_in_osds":8,"osd_in_since":1726960270,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":34},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+remapped","count":1},{"state_name":"active+recovery_wait+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1650,"data_bytes":1024950986,"bytes_used":787341312,"bytes_avail":772306771968,"bytes_total":773094113280,"degraded_objects":696,"degraded_total":3300,"degraded_ratio":0.21090909090909091,"misplaced_objects":165,"misplaced_total":3300,"misplaced_ratio":0.050000000000000003,"recovering_objects_per_sec":8,"recovering_bytes_per_sec":299793,"recovering_keys_per_sec":0,"num_objects_recovered":35,"num_bytes_recovered":1199468,"num_keys_recovered":0,"read_bytes_sec":157657,"write_bytes_sec":7678,"read_op_per_sec":180,"write_op_per_sec":30},"fsmap":{"epoch":1,"btime":"2024-09-21T23:09:25:628178+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-21T23:12:00.525386+0000","services":{}},"progress_events":{}} 2024-09-21T23:12:27.808 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T23:12:27.808 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:12:27.811 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123559 3.1a 3.a34d735a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-21T23:12:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:27.629435+0000", 2024-09-21T23:12:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.18145357200000001, 2024-09-21T23:12:27.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.181497081, 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for peered", 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123559 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629435+0000", 2024-09-21T23:12:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629435+0000", 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629436+0000", 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3549999999999999e-06 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629441+0000", 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9869999999999999e-06 2024-09-21T23:12:27.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629454+0000", 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2451e-05 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629461+0000", 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3100000000000003e-06 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:27.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629542+0000", 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0946999999999996e-05 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.629554+0000", 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2568000000000001e-05 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T23:12:27.816 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:27.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:27.853 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:27.849+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:27.941 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:27.940+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:27.945 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:27.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:28.045 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:28.044+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 86.728572890999999, 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.121 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:28.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:12:28.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:28.125 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258608022000004, 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258584866999996, 2024-09-21T23:12:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258562650000002, 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:12:28.133 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:12:28.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:28.137 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258542039000005, 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258522415000002, 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.141 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:12:28.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:12:28.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258501477999999, 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.145 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:12:28.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258479515000005, 2024-09-21T23:12:28.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:12:28.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258460091000003, 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.153 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:28.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258440949999994, 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:12:28.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:12:28.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258414500000001, 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:12:28.161 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:28.165 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258394980000006, 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258372174000002, 2024-09-21T23:12:28.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:12:28.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258351701999999, 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.173 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:28.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:12:28.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258275936999993, 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.177 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.258252264999996, 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.181 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:12:28.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:12:28.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:12:28.184 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.253337094000003, 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.185 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:12:28.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.253312367999996, 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.253287792999998, 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:12:28.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123558 3.9 3:93c04306:::rbd_data.11495bdc1465.0000000000000026:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:12:28.196 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.372338327, 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88255402500000002, 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123558 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746441+0000", 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0530000000000001e-06 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746450+0000", 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6889999999999993e-06 2024-09-21T23:12:28.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746471+0000", 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1005999999999998e-05 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746485+0000", 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746627+0000", 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014169399999999999 2024-09-21T23:12:28.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746638+0000", 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1311000000000001e-05 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621095+0000", 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87445676999999999 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621273+0000", 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017780399999999999 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2", 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621358+0000", 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5068999999999995e-05 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.623621+0000", 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022634690000000002 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628925+0000", 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053032009999999996 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:28.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628961+0000", 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5936000000000003e-05 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628993+0000", 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2274000000000003e-05 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:28.202 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:28.203 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:28.204 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:12:28.205 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:12:28.205 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:28.200+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:28.205 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:28.206 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-21T23:12:27.609233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1024950986,"num_objects":1650,"num_object_clones":294,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":963,"num_read":178579,"num_read_kb":834299,"num_write":21275,"num_write_kb":402032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3568,"num_bytes_recovered":228071938,"num_keys_recovered":600,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19067,"ondisk_log_size":19067,"up":82,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":768888,"kb_used_data":406724,"kb_used_omap":25,"kb_used_meta":361958,"kb_avail":754205832,"statfs":{"total":773094113280,"available":772306771968,"internally_reserved":0,"allocated":416485376,"data_stored":434886262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":370645039},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-107280653,"num_objects":-84,"num_object_clones":-34,"num_object_copies":-168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":-84,"num_whiteouts":-19,"num_read":721,"num_read_kb":616,"num_write":122,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1199468,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.036478"},"pg_stats":[{"pgid":"3.1f","version":"136'557","reported_seq":3959,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400095+0000","last_change":"2024-09-21T23:12:23.575904+0000","last_active":"2024-09-21T23:12:25.400095+0000","last_peered":"2024-09-21T23:12:25.400095+0000","last_clean":"2024-09-21T23:12:25.400095+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:12:25.400095+0000","last_undegraded":"2024-09-21T23:12:25.400095+0000","last_fullsized":"2024-09-21T23:12:25.400095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:06:35.248845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017863670000000001,"stat_sum":{"num_bytes":24955582,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":30,"num_read":6330,"num_read_kb":19102,"num_write":537,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1e","version":"141'829","reported_seq":3874,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.394718+0000","last_change":"2024-09-21T23:12:23.579643+0000","last_active":"2024-09-21T23:12:25.394718+0000","last_peered":"2024-09-21T23:12:25.394718+0000","last_clean":"2024-09-21T23:12:25.394718+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:12:25.394718+0000","last_undegraded":"2024-09-21T23:12:25.394718+0000","last_fullsized":"2024-09-21T23:12:25.394718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:38:50.812249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039144449999999999,"stat_sum":{"num_bytes":39916161,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":5425,"num_read_kb":41104,"num_write":1105,"num_write_kb":19714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1d","version":"144'412","reported_seq":3070,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.406573+0000","last_change":"2024-09-21T23:12:23.592278+0000","last_active":"2024-09-21T23:12:25.406573+0000","last_peered":"2024-09-21T23:12:25.406573+0000","last_clean":"2024-09-21T23:12:25.406573+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:12:25.406573+0000","last_undegraded":"2024-09-21T23:12:25.406573+0000","last_fullsized":"2024-09-21T23:12:25.406573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:52.418087+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.010589256999999999,"stat_sum":{"num_bytes":33006114,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":3058,"num_read_kb":13572,"num_write":352,"num_write_kb":11738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1c","version":"144'424","reported_seq":5330,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.519289+0000","last_change":"2024-09-21T23:12:23.583173+0000","last_active":"2024-09-21T23:12:25.519289+0000","last_peered":"2024-09-21T23:12:25.519289+0000","last_clean":"2024-09-21T23:12:25.519289+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:12:25.519289+0000","last_undegraded":"2024-09-21T23:12:25.519289+0000","last_fullsized":"2024-09-21T23:12:25.519289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:54:49.898758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001457943,"stat_sum":{"num_bytes":24933528,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":5026,"num_read_kb":22250,"num_write":448,"num_write_kb":12624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1b","version":"144'534","reported_seq":4011,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.557586+0000","last_change":"2024-09-21T23:12:23.591752+0000","last_active":"2024-09-21T23:12:25.557586+0000","last_peered":"2024-09-21T23:12:25.557586+0000","last_clean":"2024-09-21T23:12:25.557586+0000","last_became_active":"2024-09-21T23:11:07.389449+0000","last_became_peered":"2024-09-21T23:11:07.389449+0000","last_unstale":"2024-09-21T23:12:25.557586+0000","last_undegraded":"2024-09-21T23:12:25.557586+0000","last_fullsized":"2024-09-21T23:12:25.557586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:33.269776+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015889197000000001,"stat_sum":{"num_bytes":20680893,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":32,"num_read":4967,"num_read_kb":27575,"num_write":639,"num_write_kb":17922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.1a","version":"140'1197","reported_seq":7075,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.500800+0000","last_change":"2024-09-21T23:12:25.474758+0000","last_active":"2024-09-21T23:12:25.500800+0000","last_peered":"2024-09-21T23:12:25.500800+0000","last_clean":"2024-09-21T23:12:22.791916+0000","last_became_active":"2024-09-21T23:12:25.472859+0000","last_became_peered":"2024-09-21T23:12:25.472859+0000","last_unstale":"2024-09-21T23:12:25.500800+0000","last_undegraded":"2024-09-21T23:12:25.396632+0000","last_fullsized":"2024-09-21T23:12:25.392722+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1197,"log_dups_size":0,"ondisk_log_size":1197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084651799999999999,"stat_sum":{"num_bytes":25351706,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":36,"num_read":11236,"num_read_kb":29005,"num_write":1596,"num_write_kb":9623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":7665879,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":65}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.19","version":"142'950","reported_seq":2851,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.391716+0000","last_change":"2024-09-21T23:12:23.587958+0000","last_active":"2024-09-21T23:12:25.391716+0000","last_peered":"2024-09-21T23:12:25.391716+0000","last_clean":"2024-09-21T23:12:25.391716+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:12:25.391716+0000","last_undegraded":"2024-09-21T23:12:25.391716+0000","last_fullsized":"2024-09-21T23:12:25.391716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:19:45.654621+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0075067440000000001,"stat_sum":{"num_bytes":18634995,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":31,"num_read":6236,"num_read_kb":21906,"num_write":1413,"num_write_kb":13868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.18","version":"144'1005","reported_seq":5232,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.547471+0000","last_change":"2024-09-21T23:12:23.596397+0000","last_active":"2024-09-21T23:12:25.547471+0000","last_peered":"2024-09-21T23:12:25.547471+0000","last_clean":"2024-09-21T23:12:25.547471+0000","last_became_active":"2024-09-21T23:10:52.807974+0000","last_became_peered":"2024-09-21T23:10:52.807974+0000","last_unstale":"2024-09-21T23:12:25.547471+0000","last_undegraded":"2024-09-21T23:12:25.547471+0000","last_fullsized":"2024-09-21T23:12:25.547471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:11:14.363064+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014775948000000001,"stat_sum":{"num_bytes":27400001,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":8064,"num_read_kb":28905,"num_write":1405,"num_write_kb":9717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3330203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.17","version":"142'605","reported_seq":5438,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400283+0000","last_change":"2024-09-21T23:12:23.595807+0000","last_active":"2024-09-21T23:12:25.400283+0000","last_peered":"2024-09-21T23:12:25.400283+0000","last_clean":"2024-09-21T23:12:25.400283+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:12:25.400283+0000","last_undegraded":"2024-09-21T23:12:25.400283+0000","last_fullsized":"2024-09-21T23:12:25.400283+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:19:05.865092+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018468425,"stat_sum":{"num_bytes":49521309,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":40,"num_read":6004,"num_read_kb":23039,"num_write":579,"num_write_kb":12193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.16","version":"140'470","reported_seq":4209,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.402173+0000","last_change":"2024-09-21T23:12:23.582175+0000","last_active":"2024-09-21T23:12:25.402173+0000","last_peered":"2024-09-21T23:12:25.402173+0000","last_clean":"2024-09-21T23:12:25.402173+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:12:25.402173+0000","last_undegraded":"2024-09-21T23:12:25.402173+0000","last_fullsized":"2024-09-21T23:12:25.402173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:50.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095586499999999997,"stat_sum":{"num_bytes":35841524,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":30,"num_read":5486,"num_read_kb":29495,"num_write":392,"num_write_kb":12526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.15","version":"142'460","reported_seq":6428,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.396907+0000","last_change":"2024-09-21T23:12:23.594752+0000","last_active":"2024-09-21T23:12:25.396907+0000","last_peered":"2024-09-21T23:12:25.396907+0000","last_clean":"2024-09-21T23:12:25.396907+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:12:25.396907+0000","last_undegraded":"2024-09-21T23:12:25.396907+0000","last_fullsized":"2024-09-21T23:12:25.396907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:58:42.593943+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015044037,"stat_sum":{"num_bytes":60921442,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":26,"num_read":6990,"num_read_kb":36195,"num_write":399,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.14","version":"136'648","reported_seq":4713,"reported_epoch":144,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-21T23:12:25.468450+0000","last_change":"2024-09-21T23:12:25.468450+0000","last_active":"2024-09-21T23:12:25.468450+0000","last_peered":"2024-09-21T23:12:25.468450+0000","last_clean":"2024-09-21T23:12:17.521065+0000","last_became_active":"2024-09-21T23:12:25.465572+0000","last_became_peered":"2024-09-21T23:12:25.465572+0000","last_unstale":"2024-09-21T23:12:25.468450+0000","last_undegraded":"2024-09-21T23:12:25.468450+0000","last_fullsized":"2024-09-21T23:12:25.468450+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002540123,"stat_sum":{"num_bytes":12681967,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":136,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":29,"num_read":5325,"num_read_kb":13911,"num_write":789,"num_write_kb":8285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":10037734,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":44}],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399605+0000","last_change":"2024-09-21T23:11:11.428751+0000","last_active":"2024-09-21T23:12:25.399605+0000","last_peered":"2024-09-21T23:12:25.399605+0000","last_clean":"2024-09-21T23:12:25.399605+0000","last_became_active":"2024-09-21T23:11:11.427806+0000","last_became_peered":"2024-09-21T23:11:11.427806+0000","last_unstale":"2024-09-21T23:12:25.399605+0000","last_undegraded":"2024-09-21T23:12:25.399605+0000","last_fullsized":"2024-09-21T23:12:25.399605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:40.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"144'567","reported_seq":2538,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398529+0000","last_change":"2024-09-21T23:12:23.595922+0000","last_active":"2024-09-21T23:12:25.398529+0000","last_peered":"2024-09-21T23:12:25.398529+0000","last_clean":"2024-09-21T23:12:25.398529+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:12:25.398529+0000","last_undegraded":"2024-09-21T23:12:25.398529+0000","last_fullsized":"2024-09-21T23:12:25.398529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:33:08.254017+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.021543285999999998,"stat_sum":{"num_bytes":51553571,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":32,"num_read":3092,"num_read_kb":25059,"num_write":596,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392423+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:12:25.392423+0000","last_peered":"2024-09-21T23:12:25.392423+0000","last_clean":"2024-09-21T23:12:25.392423+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:12:25.392423+0000","last_undegraded":"2024-09-21T23:12:25.392423+0000","last_fullsized":"2024-09-21T23:12:25.392423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"139'735","reported_seq":5701,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395364+0000","last_change":"2024-09-21T23:12:23.579231+0000","last_active":"2024-09-21T23:12:25.395364+0000","last_peered":"2024-09-21T23:12:25.395364+0000","last_clean":"2024-09-21T23:12:25.395364+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:12:25.395364+0000","last_undegraded":"2024-09-21T23:12:25.395364+0000","last_fullsized":"2024-09-21T23:12:25.395364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:15:27.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037482779999999999,"stat_sum":{"num_bytes":59845519,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":24,"num_read":7997,"num_read_kb":30135,"num_write":1011,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398221+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:12:25.398221+0000","last_peered":"2024-09-21T23:12:25.398221+0000","last_clean":"2024-09-21T23:12:25.398221+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:12:25.398221+0000","last_undegraded":"2024-09-21T23:12:25.398221+0000","last_fullsized":"2024-09-21T23:12:25.398221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"144'609","reported_seq":3657,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.548797+0000","last_change":"2024-09-21T23:12:23.591709+0000","last_active":"2024-09-21T23:12:25.548797+0000","last_peered":"2024-09-21T23:12:25.548797+0000","last_clean":"2024-09-21T23:12:25.548797+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:12:25.548797+0000","last_undegraded":"2024-09-21T23:12:25.548797+0000","last_fullsized":"2024-09-21T23:12:25.548797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:19.665608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097742330000000002,"stat_sum":{"num_bytes":23242291,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":5031,"num_read_kb":24018,"num_write":567,"num_write_kb":10680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392128+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:12:25.392128+0000","last_peered":"2024-09-21T23:12:25.392128+0000","last_clean":"2024-09-21T23:12:25.392128+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:12:25.392128+0000","last_undegraded":"2024-09-21T23:12:25.392128+0000","last_fullsized":"2024-09-21T23:12:25.392128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"141'761","reported_seq":5770,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.407237+0000","last_change":"2024-09-21T23:12:23.584716+0000","last_active":"2024-09-21T23:12:25.407237+0000","last_peered":"2024-09-21T23:12:25.407237+0000","last_clean":"2024-09-21T23:12:25.407237+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:12:25.407237+0000","last_undegraded":"2024-09-21T23:12:25.407237+0000","last_fullsized":"2024-09-21T23:12:25.407237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:27.235272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002790463,"stat_sum":{"num_bytes":60691725,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":33,"num_read":6756,"num_read_kb":41318,"num_write":1076,"num_write_kb":18307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.3","version":"142'660","reported_seq":4056,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.393364+0000","last_change":"2024-09-21T23:12:23.573763+0000","last_active":"2024-09-21T23:12:25.393364+0000","last_peered":"2024-09-21T23:12:25.393364+0000","last_clean":"2024-09-21T23:12:25.393364+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:12:25.393364+0000","last_undegraded":"2024-09-21T23:12:25.393364+0000","last_fullsized":"2024-09-21T23:12:25.393364+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:16:31.132834+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041488519999999997,"stat_sum":{"num_bytes":28607623,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":5511,"num_read_kb":26445,"num_write":659,"num_write_kb":12055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.2","version":"20'2","reported_seq":270,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400665+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:12:25.400665+0000","last_peered":"2024-09-21T23:12:25.400665+0000","last_clean":"2024-09-21T23:12:25.400665+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:12:25.400665+0000","last_undegraded":"2024-09-21T23:12:25.400665+0000","last_fullsized":"2024-09-21T23:12:25.400665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"142'532","reported_seq":4439,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.391720+0000","last_change":"2024-09-21T23:12:23.594569+0000","last_active":"2024-09-21T23:12:25.391720+0000","last_peered":"2024-09-21T23:12:25.391720+0000","last_clean":"2024-09-21T23:12:25.391720+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:12:25.391720+0000","last_undegraded":"2024-09-21T23:12:25.391720+0000","last_fullsized":"2024-09-21T23:12:25.391720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:03:19.335888+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0138751,"stat_sum":{"num_bytes":23945683,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":32,"num_read":6639,"num_read_kb":20514,"num_write":470,"num_write_kb":13136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":267,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399873+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:12:25.399873+0000","last_peered":"2024-09-21T23:12:25.399873+0000","last_clean":"2024-09-21T23:12:25.399873+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:12:25.399873+0000","last_undegraded":"2024-09-21T23:12:25.399873+0000","last_fullsized":"2024-09-21T23:12:25.399873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"144'439","reported_seq":5223,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.459920+0000","last_change":"2024-09-21T23:12:25.459920+0000","last_active":"2024-09-21T23:12:25.459920+0000","last_peered":"2024-09-21T23:12:25.459920+0000","last_clean":"2024-09-21T23:12:22.803009+0000","last_became_active":"2024-09-21T23:12:25.458543+0000","last_became_peered":"2024-09-21T23:12:25.458543+0000","last_unstale":"2024-09-21T23:12:25.459920+0000","last_undegraded":"2024-09-21T23:12:25.400767+0000","last_fullsized":"2024-09-21T23:12:25.398785+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074971900000000004,"stat_sum":{"num_bytes":29727255,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":30,"num_read":4756,"num_read_kb":20310,"num_write":488,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":6739606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,3","objects":46}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":140,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395082+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:12:25.395082+0000","last_peered":"2024-09-21T23:12:25.395082+0000","last_clean":"2024-09-21T23:12:25.395082+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:12:25.395082+0000","last_undegraded":"2024-09-21T23:12:25.395082+0000","last_fullsized":"2024-09-21T23:12:25.395082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"141'541","reported_seq":4182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395824+0000","last_change":"2024-09-21T23:12:23.578066+0000","last_active":"2024-09-21T23:12:25.395824+0000","last_peered":"2024-09-21T23:12:25.395824+0000","last_clean":"2024-09-21T23:12:25.395824+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:12:25.395824+0000","last_undegraded":"2024-09-21T23:12:25.395824+0000","last_fullsized":"2024-09-21T23:12:25.395824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:23.184287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032415339999999999,"stat_sum":{"num_bytes":6317009,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":5243,"num_read_kb":19488,"num_write":626,"num_write_kb":14848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"2.3","version":"18'1","reported_seq":269,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.400115+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:12:25.400115+0000","last_peered":"2024-09-21T23:12:25.400115+0000","last_clean":"2024-09-21T23:12:25.400115+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:12:25.400115+0000","last_undegraded":"2024-09-21T23:12:25.400115+0000","last_fullsized":"2024-09-21T23:12:25.400115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":366,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.399214+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:12:25.399214+0000","last_peered":"2024-09-21T23:12:25.399214+0000","last_clean":"2024-09-21T23:12:25.399214+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:12:25.399214+0000","last_undegraded":"2024-09-21T23:12:25.399214+0000","last_fullsized":"2024-09-21T23:12:25.399214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"142'474","reported_seq":3347,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.392920+0000","last_change":"2024-09-21T23:12:23.581501+0000","last_active":"2024-09-21T23:12:25.392920+0000","last_peered":"2024-09-21T23:12:25.392920+0000","last_clean":"2024-09-21T23:12:25.392920+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:12:25.392920+0000","last_undegraded":"2024-09-21T23:12:25.392920+0000","last_fullsized":"2024-09-21T23:12:25.392920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:23:33.737770+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0066331569999999998,"stat_sum":{"num_bytes":5864293,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":4360,"num_read_kb":9771,"num_write":470,"num_write_kb":5012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.9","version":"140'473","reported_seq":3988,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.457106+0000","last_change":"2024-09-21T23:12:25.457459+0000","last_active":"2024-09-21T23:12:25.457106+0000","last_peered":"2024-09-21T23:12:25.457106+0000","last_clean":"2024-09-21T23:12:22.773644+0000","last_became_active":"2024-09-21T23:12:25.456530+0000","last_became_peered":"2024-09-21T23:12:25.456530+0000","last_unstale":"2024-09-21T23:12:25.457106+0000","last_undegraded":"2024-09-21T23:12:25.399079+0000","last_fullsized":"2024-09-21T23:12:25.395974+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053112500000000002,"stat_sum":{"num_bytes":22697745,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":5637,"num_read_kb":19185,"num_write":443,"num_write_kb":5926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":4244532,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[2],"avail_no_missing":["2","7"],"object_location_counts":[{"shards":"2,7","objects":45}],"blocked_by":[],"up_primary":4,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.a","version":"140'568","reported_seq":5987,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.462692+0000","last_change":"2024-09-21T23:12:25.463123+0000","last_active":"2024-09-21T23:12:25.462692+0000","last_peered":"2024-09-21T23:12:25.462692+0000","last_clean":"2024-09-21T23:12:22.780649+0000","last_became_active":"2024-09-21T23:12:25.460731+0000","last_became_peered":"2024-09-21T23:12:25.460731+0000","last_unstale":"2024-09-21T23:12:25.462692+0000","last_undegraded":"2024-09-21T23:12:25.405385+0000","last_fullsized":"2024-09-21T23:12:25.401307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016923999999999999,"stat_sum":{"num_bytes":28480289,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":35,"num_read":6682,"num_read_kb":40112,"num_write":509,"num_write_kb":11929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":16052830,"num_keys_recovered":97,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":["6","1"],"object_location_counts":[{"shards":"1,6","objects":63}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.b","version":"138'353","reported_seq":3407,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.393434+0000","last_change":"2024-09-21T23:12:23.578585+0000","last_active":"2024-09-21T23:12:25.393434+0000","last_peered":"2024-09-21T23:12:25.393434+0000","last_clean":"2024-09-21T23:12:25.393434+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:12:25.393434+0000","last_undegraded":"2024-09-21T23:12:25.393434+0000","last_fullsized":"2024-09-21T23:12:25.393434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:27:01.335640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035367049999999998,"stat_sum":{"num_bytes":22020270,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":3313,"num_read_kb":26228,"num_write":320,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.c","version":"139'494","reported_seq":2095,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.401637+0000","last_change":"2024-09-21T23:12:23.582319+0000","last_active":"2024-09-21T23:12:25.401637+0000","last_peered":"2024-09-21T23:12:25.401637+0000","last_clean":"2024-09-21T23:12:25.401637+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:12:25.401637+0000","last_undegraded":"2024-09-21T23:12:25.401637+0000","last_fullsized":"2024-09-21T23:12:25.401637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:30:54.665289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011079219999999999,"stat_sum":{"num_bytes":19109519,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":38,"num_read":2206,"num_read_kb":17676,"num_write":471,"num_write_kb":9186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.d","version":"142'500","reported_seq":5026,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395926+0000","last_change":"2024-09-21T23:12:23.584209+0000","last_active":"2024-09-21T23:12:25.395926+0000","last_peered":"2024-09-21T23:12:25.395926+0000","last_clean":"2024-09-21T23:12:25.395926+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:12:25.395926+0000","last_undegraded":"2024-09-21T23:12:25.395926+0000","last_fullsized":"2024-09-21T23:12:25.395926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:52:18.266627+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070675219999999997,"stat_sum":{"num_bytes":53356632,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":38,"num_read":5085,"num_read_kb":28814,"num_write":425,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.e","version":"141'457","reported_seq":4530,"reported_epoch":144,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:12:25.467578+0000","last_change":"2024-09-21T23:12:25.468015+0000","last_active":"2024-09-21T23:12:25.467578+0000","last_peered":"2024-09-21T23:12:25.467578+0000","last_clean":"2024-09-21T23:12:22.697992+0000","last_became_active":"2024-09-21T23:12:25.465677+0000","last_became_peered":"2024-09-21T23:12:25.465677+0000","last_unstale":"2024-09-21T23:12:25.467578+0000","last_undegraded":"2024-09-21T23:12:25.400587+0000","last_fullsized":"2024-09-21T23:12:25.395088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028850960000000002,"stat_sum":{"num_bytes":47211785,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":35,"num_read":6688,"num_read_kb":23677,"num_write":456,"num_write_kb":9490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":16260526,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"1,7","objects":56}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1e","length":"2"}]},{"pgid":"3.f","version":"144'537","reported_seq":2374,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.396129+0000","last_change":"2024-09-21T23:12:23.589358+0000","last_active":"2024-09-21T23:12:25.396129+0000","last_peered":"2024-09-21T23:12:25.396129+0000","last_clean":"2024-09-21T23:12:25.396129+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:12:25.396129+0000","last_undegraded":"2024-09-21T23:12:25.396129+0000","last_fullsized":"2024-09-21T23:12:25.396129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:01:53.460212+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010235556,"stat_sum":{"num_bytes":17603506,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":38,"num_read":2338,"num_read_kb":21657,"num_write":593,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.10","version":"142'610","reported_seq":3880,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.397340+0000","last_change":"2024-09-21T23:12:23.596504+0000","last_active":"2024-09-21T23:12:25.397340+0000","last_peered":"2024-09-21T23:12:25.397340+0000","last_clean":"2024-09-21T23:12:25.397340+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:12:25.397340+0000","last_undegraded":"2024-09-21T23:12:25.397340+0000","last_fullsized":"2024-09-21T23:12:25.397340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:49.322900+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013222350000000001,"stat_sum":{"num_bytes":50187387,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":5870,"num_read_kb":37397,"num_write":642,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.11","version":"142'616","reported_seq":6748,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.395599+0000","last_change":"2024-09-21T23:12:23.588449+0000","last_active":"2024-09-21T23:12:25.395599+0000","last_peered":"2024-09-21T23:12:25.395599+0000","last_clean":"2024-09-21T23:12:25.395599+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:12:25.395599+0000","last_undegraded":"2024-09-21T23:12:25.395599+0000","last_fullsized":"2024-09-21T23:12:25.395599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:24.702332+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.011401987000000001,"stat_sum":{"num_bytes":60771037,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":7995,"num_read_kb":42900,"num_write":653,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.12","version":"142'546","reported_seq":6812,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T23:12:25.398361+0000","last_change":"2024-09-21T23:12:23.578594+0000","last_active":"2024-09-21T23:12:25.398361+0000","last_peered":"2024-09-21T23:12:25.398361+0000","last_clean":"2024-09-21T23:12:25.398361+0000","last_became_active":"2024-09-21T23:11:15.382800+0000","last_became_peered":"2024-09-21T23:11:15.382800+0000","last_unstale":"2024-09-21T23:12:25.398361+0000","last_undegraded":"2024-09-21T23:12:25.398361+0000","last_fullsized":"2024-09-21T23:12:25.398361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:55.305903+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0044734670000000001,"stat_sum":{"num_bytes":21979199,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":7663,"num_read_kb":39292,"num_write":614,"num_write_kb":21738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"3"}]},{"pgid":"3.13","version":"141'384","reported_seq":2162,"reported_epoch":144,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-21T23:12:25.551458+0000","last_change":"2024-09-21T23:12:25.465200+0000","last_active":"2024-09-21T23:12:25.551458+0000","last_peered":"2024-09-21T23:12:25.551458+0000","last_clean":"2024-09-21T23:12:22.713475+0000","last_became_active":"2024-09-21T23:12:25.446879+0000","last_became_peered":"2024-09-21T23:12:25.446879+0000","last_unstale":"2024-09-21T23:12:25.551458+0000","last_undegraded":"2024-09-21T23:12:25.551458+0000","last_fullsized":"2024-09-21T23:12:25.395801+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002440593,"stat_sum":{"num_bytes":17303039,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":33,"num_read":1454,"num_read_kb":14144,"num_write":371,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":19217578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","2"],"object_location_counts":[{"shards":"2,7","objects":29},{"shards":"7","objects":22}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1e","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1024360599,"num_objects":1646,"num_object_clones":294,"num_object_copies":3292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":696,"num_objects_misplaced":165,"num_objects_unfound":0,"num_objects_dirty":1646,"num_whiteouts":963,"num_read":178463,"num_read_kb":834199,"num_write":21114,"num_write_kb":399614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3564,"num_bytes_recovered":226891202,"num_keys_recovered":600,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401350656,"data_stored":426810805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":18947,"ondisk_log_size":18947,"up":64,"acting":58,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124892,"kb_used_data":71496,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94246948,"statfs":{"total":96636764160,"available":96508874752,"internally_reserved":0,"allocated":73211904,"data_stored":74382176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70260,"kb_used_data":30744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94301580,"statfs":{"total":96636764160,"available":96564817920,"internally_reserved":0,"allocated":31481856,"data_stored":31064524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":50644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":51859456,"data_stored":58668357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743442,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66224,"kb_used_data":37592,"kb_used_omap":15,"kb_used_meta":28592,"kb_avail":94305616,"statfs":{"total":96636764160,"available":96568950784,"internally_reserved":0,"allocated":38494208,"data_stored":40217465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":29279170},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140912,"kb_used_data":69468,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94230928,"statfs":{"total":96636764160,"available":96492470272,"internally_reserved":0,"allocated":71135232,"data_stored":73991323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68432,"kb_used_data":28920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94303408,"statfs":{"total":96636764160,"available":96566689792,"internally_reserved":0,"allocated":29614080,"data_stored":30458988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97236,"kb_used_data":48000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94274604,"statfs":{"total":96636764160,"available":96537194496,"internally_reserved":0,"allocated":49152000,"data_stored":50324918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123260,"kb_used_data":69860,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94248580,"statfs":{"total":96636764160,"available":96510545920,"internally_reserved":0,"allocated":71536640,"data_stored":75778511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69091328,"data_stored":74163552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":47431680,"data_stored":49496954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27885568,"data_stored":29624898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69419008,"data_stored":73163378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36917248,"data_stored":39517697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50020352,"data_stored":57715581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29765632,"data_stored":30236579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70819840,"data_stored":72892166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:12:28.304 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:28.301+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:28.372 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:28.372 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:28.372 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:28.372 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:28.388 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:28.401 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:28.401 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:28.418 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:28.418 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:28.418 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:28.413+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:28.435 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:28.435 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:28.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:28.451 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:28.472 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:28.472 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:28.492 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:28.492 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:28.506 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:28.504+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:28.508 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:28.508 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:28.512 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:28.513 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:28.520 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:28.534 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:28.534 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:28.606 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:28.604+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:28.665 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:28.665 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:28.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:28.707 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:28.704+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:28.796 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:28.796 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:28.808 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:28.809 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:28.804+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:28.909 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:28.905+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:28.928 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:28.928 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:28.942 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:29.009 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:29.008+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:29.106 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:29.106 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:29.110 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:29.108+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:29.120 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:29.211 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:29.205+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:29.237 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:29.237 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:29.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:29.312 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:29.308+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:29.409 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:29.409 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:29.413 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:29.412+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:29.423 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:29.513 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:29.512+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:29.545 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:29.546 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:29.561 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:29.614 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:29.609+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:29.640 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:12:29.688 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:29.688 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:29.706 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:29.715 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:29.712+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:29.816 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:29.812+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:29.840 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:29.840 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:29.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:29.917 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:29.916+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:29.970 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:29.970 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:29.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:30.017 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:30.013+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:30.116 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:30.117 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:30.118 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:30.116+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:30.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:30.219 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:30.213+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:30.296 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:30.296 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:30.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:30.319 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:30.317+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:30.403 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:12:30.419 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:30.413+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:30.449 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:30.449 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:30.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:30.520 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:30.516+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:30.551 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:30.591 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:30.592 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:30.606 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:30.620 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:30.617+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:30.704 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:30.704 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:30.721 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:30.721 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:30.722 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:30.717+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:30.737 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:30.737 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:30.754 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:30.754 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:30.764 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:30.764 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:30.774 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:30.774 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:30.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:30.792 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:30.792 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:30.810 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:30.810 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:30.825 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:30.824+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:30.828 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:30.829 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 112.174986511, 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.906 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:12:30.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:12:30.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:12:30.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:12:30.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527673018000002, 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.913 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:12:30.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527640343000002, 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.917 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:30.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:12:30.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:12:30.920 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527617007999993, 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:12:30.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:12:30.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:12:30.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527584442999995, 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.925 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:30.929 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527496821, 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:12:30.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:30.933 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527378073999998, 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527353309000006, 2024-09-21T23:12:30.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527332161999993, 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:12:30.941 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:12:30.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527274465000005, 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:30.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527239584, 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:12:30.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:12:30.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.953 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527214059000002, 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527142873000003, 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:30.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.961 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527107027, 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:12:30.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527089934000003, 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.966 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527073923000003, 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.970 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.527016674999999, 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.974 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:30.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:12:30.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:12:30.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.526935309999999, 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.978 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.526903759999996, 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:12:30.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:12:30.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.526756863000003, 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:30.987 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:30.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:30.991 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:30.992 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:30.993 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:30.988+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:31.093 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:31.092+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:31.163 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:31.163 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:31.163 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:31.164 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:31.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:31.209 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:12:31.221 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:31.220+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:31.296 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:31.292+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:31.342 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:31.342 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:31.342 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:31.342 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:31.342 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:31.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:31.396 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:31.389+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:31.497 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:31.496+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:31.523 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.337125622, 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:31.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:12:31.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:12:31.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.33702253600001, 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:31.531 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.336955187, 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.26176825100001, 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.261712483, 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.539 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:12:31.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.26168094400001, 2024-09-21T23:12:31.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:12:31.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:12:31.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.547 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.26164819499999, 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:31.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:12:31.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.26162130200001, 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.261596029, 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.556 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:12:31.559 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.261564902, 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.560 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.26092154599999, 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.564 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:12:31.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:12:31.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.25571413599999, 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.568 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:31.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:31.572 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.25568985000001, 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:12:31.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.255672183, 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.577 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:12:31.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.25564859799999, 2024-09-21T23:12:31.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:12:31.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:12:31.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.25563258299999, 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:12:31.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:12:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.255618752, 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.590 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.594 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.25558739100001, 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:12:31.598 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 144.25551896900001, 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123547 3.13 3:c9e2c0b1:::rbd_data.11495bdc1465.000000000000000d:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 4.77883148, 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88065156200000005, 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:31.603 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123547 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:31.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742119+0000", 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0244e-05 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742127+0000", 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6100000000000006e-06 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742154+0000", 2024-09-21T23:12:31.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6730999999999999e-05 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742165+0000", 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1419e-05 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742241+0000", 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5562999999999995e-05 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:31.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742251+0000", 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0017000000000001e-05 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619014+0000", 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87676288899999999 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619157+0000", 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143549 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 4", 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619280+0000", 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012234100000000001 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621009+0000", 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001729511 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622618+0000", 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001609262 2024-09-21T23:12:31.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622687+0000", 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8813000000000007e-05 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622740+0000", 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2612999999999999e-05 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:31.609 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:12:31.612 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:12:31.613 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:31.613 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-21T23:12:31.401926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":968934058,"num_objects":1414,"num_object_clones":203,"num_object_copies":2828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1414,"num_whiteouts":880,"num_read":179697,"num_read_kb":835251,"num_write":21444,"num_write_kb":402074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4503,"num_bytes_recovered":252919275,"num_keys_recovered":713,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19429,"ondisk_log_size":19429,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":761904,"kb_used_data":378828,"kb_used_omap":25,"kb_used_meta":382758,"kb_avail":754212816,"statfs":{"total":773094113280,"available":772313923584,"internally_reserved":0,"allocated":387919872,"data_stored":405664790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":391944239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-56016928,"num_objects":-236,"num_object_clones":-91,"num_object_copies":-472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-696,"num_objects_misplaced":-165,"num_objects_unfound":0,"num_objects_dirty":-236,"num_whiteouts":-83,"num_read":1118,"num_read_kb":952,"num_write":169,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":935,"num_bytes_recovered":24847337,"num_keys_recovered":113,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.792692"},"pg_stats":[{"pgid":"3.1f","version":"149'562","reported_seq":3988,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.420380+0000","last_change":"2024-09-21T23:12:30.420842+0000","last_active":"2024-09-21T23:12:30.420380+0000","last_peered":"2024-09-21T23:12:30.420380+0000","last_clean":"2024-09-21T23:12:30.420380+0000","last_became_active":"2024-09-21T23:09:55.376984+0000","last_became_peered":"2024-09-21T23:09:55.376984+0000","last_unstale":"2024-09-21T23:12:30.420380+0000","last_undegraded":"2024-09-21T23:12:30.420380+0000","last_fullsized":"2024-09-21T23:12:30.420380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:35.892994+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.025162421000000001,"stat_sum":{"num_bytes":23423678,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":28,"num_read":6330,"num_read_kb":19102,"num_write":538,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1e","version":"149'836","reported_seq":3907,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.413180+0000","last_change":"2024-09-21T23:12:30.413898+0000","last_active":"2024-09-21T23:12:30.413180+0000","last_peered":"2024-09-21T23:12:30.413180+0000","last_clean":"2024-09-21T23:12:30.413180+0000","last_became_active":"2024-09-21T23:09:55.379173+0000","last_became_peered":"2024-09-21T23:09:55.379173+0000","last_unstale":"2024-09-21T23:12:30.413180+0000","last_undegraded":"2024-09-21T23:12:30.413180+0000","last_fullsized":"2024-09-21T23:12:30.413180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:39:14.152158+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016492389999999999,"stat_sum":{"num_bytes":34765911,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":5428,"num_read_kb":41105,"num_write":1108,"num_write_kb":19714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1d","version":"149'420","reported_seq":3104,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.432594+0000","last_change":"2024-09-21T23:12:30.432594+0000","last_active":"2024-09-21T23:12:30.432594+0000","last_peered":"2024-09-21T23:12:30.432594+0000","last_clean":"2024-09-21T23:12:30.432594+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:12:30.432594+0000","last_undegraded":"2024-09-21T23:12:30.432594+0000","last_fullsized":"2024-09-21T23:12:30.432594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:51.693697+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.029319194999999999,"stat_sum":{"num_bytes":30365693,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":25,"num_read":3058,"num_read_kb":13572,"num_write":356,"num_write_kb":11738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1c","version":"149'430","reported_seq":5382,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.498149+0000","last_change":"2024-09-21T23:12:30.420902+0000","last_active":"2024-09-21T23:12:30.498149+0000","last_peered":"2024-09-21T23:12:30.498149+0000","last_clean":"2024-09-21T23:12:30.498149+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:12:30.498149+0000","last_undegraded":"2024-09-21T23:12:30.498149+0000","last_fullsized":"2024-09-21T23:12:30.498149+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:24:58.357130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017936127999999999,"stat_sum":{"num_bytes":24212632,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":5053,"num_read_kb":22277,"num_write":457,"num_write_kb":12624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1b","version":"149'538","reported_seq":4040,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.426903+0000","last_change":"2024-09-21T23:12:30.426903+0000","last_active":"2024-09-21T23:12:30.426903+0000","last_peered":"2024-09-21T23:12:30.426903+0000","last_clean":"2024-09-21T23:12:30.426903+0000","last_became_active":"2024-09-21T23:11:07.389449+0000","last_became_peered":"2024-09-21T23:11:07.389449+0000","last_unstale":"2024-09-21T23:12:30.426903+0000","last_undegraded":"2024-09-21T23:12:30.426903+0000","last_fullsized":"2024-09-21T23:12:30.426903+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:17.223127+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.031222327000000001,"stat_sum":{"num_bytes":18960573,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":31,"num_read":4967,"num_read_kb":27575,"num_write":641,"num_write_kb":17922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1a","version":"149'1222","reported_seq":7199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.469413+0000","last_change":"2024-09-21T23:12:30.424292+0000","last_active":"2024-09-21T23:12:30.469413+0000","last_peered":"2024-09-21T23:12:30.469413+0000","last_clean":"2024-09-21T23:12:30.469413+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:12:30.469413+0000","last_undegraded":"2024-09-21T23:12:30.469413+0000","last_fullsized":"2024-09-21T23:12:30.469413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1222,"log_dups_size":0,"ondisk_log_size":1222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:24.013174+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.029208720000000001,"stat_sum":{"num_bytes":24647150,"num_objects":57,"num_object_clones":9,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":33,"num_read":11411,"num_read_kb":29155,"num_write":1615,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.19","version":"149'955","reported_seq":2883,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.496849+0000","last_change":"2024-09-21T23:12:30.417783+0000","last_active":"2024-09-21T23:12:30.496849+0000","last_peered":"2024-09-21T23:12:30.496849+0000","last_clean":"2024-09-21T23:12:30.496849+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:12:30.496849+0000","last_undegraded":"2024-09-21T23:12:30.496849+0000","last_fullsized":"2024-09-21T23:12:30.496849+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":955,"log_dups_size":0,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:01.901291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010037399000000001,"stat_sum":{"num_bytes":13916387,"num_objects":47,"num_object_clones":6,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":30,"num_read":6239,"num_read_kb":21907,"num_write":1416,"num_write_kb":13868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.18","version":"149'1016","reported_seq":5267,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.424677+0000","last_change":"2024-09-21T23:12:30.425152+0000","last_active":"2024-09-21T23:12:30.424677+0000","last_peered":"2024-09-21T23:12:30.424677+0000","last_clean":"2024-09-21T23:12:30.424677+0000","last_became_active":"2024-09-21T23:10:52.807974+0000","last_became_peered":"2024-09-21T23:10:52.807974+0000","last_unstale":"2024-09-21T23:12:30.424677+0000","last_undegraded":"2024-09-21T23:12:30.424677+0000","last_fullsized":"2024-09-21T23:12:30.424677+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:56:29.246588+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.020962590999999999,"stat_sum":{"num_bytes":26381909,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":25,"num_read":8077,"num_read_kb":28916,"num_write":1412,"num_write_kb":9721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3330203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.17","version":"149'620","reported_seq":5475,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.446574+0000","last_change":"2024-09-21T23:12:30.446574+0000","last_active":"2024-09-21T23:12:30.446574+0000","last_peered":"2024-09-21T23:12:30.446574+0000","last_clean":"2024-09-21T23:12:30.446574+0000","last_became_active":"2024-09-21T23:09:55.373371+0000","last_became_peered":"2024-09-21T23:09:55.373371+0000","last_unstale":"2024-09-21T23:12:30.446574+0000","last_undegraded":"2024-09-21T23:12:30.446574+0000","last_fullsized":"2024-09-21T23:12:30.446574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:53.724117+0000","scrub_duration":0,"objects_trimmed":9,"snaptrim_duration":0.050230457999999999,"stat_sum":{"num_bytes":46677632,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":34,"num_read":6004,"num_read_kb":23039,"num_write":580,"num_write_kb":12193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.16","version":"149'478","reported_seq":4244,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.447333+0000","last_change":"2024-09-21T23:12:30.447333+0000","last_active":"2024-09-21T23:12:30.447333+0000","last_peered":"2024-09-21T23:12:30.447333+0000","last_clean":"2024-09-21T23:12:30.447333+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:12:30.447333+0000","last_undegraded":"2024-09-21T23:12:30.447333+0000","last_fullsized":"2024-09-21T23:12:30.447333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:17:27.684710+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.041163997000000001,"stat_sum":{"num_bytes":34616068,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":27,"num_read":5492,"num_read_kb":29498,"num_write":394,"num_write_kb":12526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.15","version":"149'471","reported_seq":6462,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.454132+0000","last_change":"2024-09-21T23:12:30.454132+0000","last_active":"2024-09-21T23:12:30.454132+0000","last_peered":"2024-09-21T23:12:30.454132+0000","last_clean":"2024-09-21T23:12:30.454132+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:12:30.454132+0000","last_undegraded":"2024-09-21T23:12:30.454132+0000","last_fullsized":"2024-09-21T23:12:30.454132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:21.369798+0000","scrub_duration":0,"objects_trimmed":9,"snaptrim_duration":0.038811545000000003,"stat_sum":{"num_bytes":59405922,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":6990,"num_read_kb":36195,"num_write":400,"num_write_kb":15021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.14","version":"149'652","reported_seq":4739,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.412167+0000","last_change":"2024-09-21T23:12:30.412167+0000","last_active":"2024-09-21T23:12:30.412167+0000","last_peered":"2024-09-21T23:12:30.412167+0000","last_clean":"2024-09-21T23:12:30.412167+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:12:30.412167+0000","last_undegraded":"2024-09-21T23:12:30.412167+0000","last_fullsized":"2024-09-21T23:12:30.412167+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:53:53.882869+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016934563999999999,"stat_sum":{"num_bytes":12347063,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":27,"num_read":5325,"num_read_kb":13911,"num_write":789,"num_write_kb":8285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":288,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.407537+0000","last_change":"2024-09-21T23:11:11.428751+0000","last_active":"2024-09-21T23:12:30.407537+0000","last_peered":"2024-09-21T23:12:30.407537+0000","last_clean":"2024-09-21T23:12:30.407537+0000","last_became_active":"2024-09-21T23:11:11.427806+0000","last_became_peered":"2024-09-21T23:11:11.427806+0000","last_unstale":"2024-09-21T23:12:30.407537+0000","last_undegraded":"2024-09-21T23:12:30.407537+0000","last_fullsized":"2024-09-21T23:12:30.407537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:40.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"149'582","reported_seq":2575,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.450466+0000","last_change":"2024-09-21T23:12:30.450466+0000","last_active":"2024-09-21T23:12:30.450466+0000","last_peered":"2024-09-21T23:12:30.450466+0000","last_clean":"2024-09-21T23:12:30.450466+0000","last_became_active":"2024-09-21T23:09:55.370068+0000","last_became_peered":"2024-09-21T23:09:55.370068+0000","last_unstale":"2024-09-21T23:12:30.450466+0000","last_undegraded":"2024-09-21T23:12:30.450466+0000","last_fullsized":"2024-09-21T23:12:30.450466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:30:47.602905+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.052772850000000003,"stat_sum":{"num_bytes":49079596,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":26,"num_read":3092,"num_read_kb":25059,"num_write":596,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.6","version":"0'0","reported_seq":277,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.408784+0000","last_change":"2024-09-21T23:09:47.726083+0000","last_active":"2024-09-21T23:12:30.408784+0000","last_peered":"2024-09-21T23:12:30.408784+0000","last_clean":"2024-09-21T23:12:30.408784+0000","last_became_active":"2024-09-21T23:09:45.711814+0000","last_became_peered":"2024-09-21T23:09:45.711814+0000","last_unstale":"2024-09-21T23:12:30.408784+0000","last_undegraded":"2024-09-21T23:12:30.408784+0000","last_fullsized":"2024-09-21T23:12:30.408784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:14.937540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"149'741","reported_seq":5736,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.422085+0000","last_change":"2024-09-21T23:12:30.422747+0000","last_active":"2024-09-21T23:12:30.422085+0000","last_peered":"2024-09-21T23:12:30.422085+0000","last_clean":"2024-09-21T23:12:30.422085+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:12:30.422085+0000","last_undegraded":"2024-09-21T23:12:30.422085+0000","last_fullsized":"2024-09-21T23:12:30.422085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:16.361255+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.024849957999999998,"stat_sum":{"num_bytes":58894965,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":8007,"num_read_kb":30140,"num_write":1013,"num_write_kb":19444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.415815+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:12:30.415815+0000","last_peered":"2024-09-21T23:12:30.415815+0000","last_clean":"2024-09-21T23:12:30.415815+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:12:30.415815+0000","last_undegraded":"2024-09-21T23:12:30.415815+0000","last_fullsized":"2024-09-21T23:12:30.415815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"149'649","reported_seq":3793,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.529670+0000","last_change":"2024-09-21T23:12:30.426659+0000","last_active":"2024-09-21T23:12:30.529670+0000","last_peered":"2024-09-21T23:12:30.529670+0000","last_clean":"2024-09-21T23:12:30.529670+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:12:30.529670+0000","last_undegraded":"2024-09-21T23:12:30.529670+0000","last_fullsized":"2024-09-21T23:12:30.529670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:00:31.008403+0000","scrub_duration":0,"objects_trimmed":9,"snaptrim_duration":0.018944863999999999,"stat_sum":{"num_bytes":19809643,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":23,"num_read":5200,"num_read_kb":24165,"num_write":590,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.4","version":"0'0","reported_seq":277,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.408273+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:12:30.408273+0000","last_peered":"2024-09-21T23:12:30.408273+0000","last_clean":"2024-09-21T23:12:30.408273+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:12:30.408273+0000","last_undegraded":"2024-09-21T23:12:30.408273+0000","last_fullsized":"2024-09-21T23:12:30.408273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"149'777","reported_seq":5814,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.442437+0000","last_change":"2024-09-21T23:12:30.444036+0000","last_active":"2024-09-21T23:12:30.442437+0000","last_peered":"2024-09-21T23:12:30.442437+0000","last_clean":"2024-09-21T23:12:30.442437+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:12:30.442437+0000","last_undegraded":"2024-09-21T23:12:30.442437+0000","last_fullsized":"2024-09-21T23:12:30.442437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:41.733015+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.039520502999999998,"stat_sum":{"num_bytes":60261574,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":31,"num_read":6787,"num_read_kb":41342,"num_write":1090,"num_write_kb":18312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.3","version":"149'680","reported_seq":4112,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.431578+0000","last_change":"2024-09-21T23:12:30.434535+0000","last_active":"2024-09-21T23:12:30.431578+0000","last_peered":"2024-09-21T23:12:30.431578+0000","last_clean":"2024-09-21T23:12:30.431578+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:12:30.431578+0000","last_undegraded":"2024-09-21T23:12:30.431578+0000","last_fullsized":"2024-09-21T23:12:30.431578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:53:20.786066+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.038963398000000003,"stat_sum":{"num_bytes":26695643,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":5529,"num_read_kb":26463,"num_write":673,"num_write_kb":12064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.2","version":"20'2","reported_seq":280,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.402357+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:12:30.402357+0000","last_peered":"2024-09-21T23:12:30.402357+0000","last_clean":"2024-09-21T23:12:30.402357+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:12:30.402357+0000","last_undegraded":"2024-09-21T23:12:30.402357+0000","last_fullsized":"2024-09-21T23:12:30.402357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"149'539","reported_seq":4470,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.416609+0000","last_change":"2024-09-21T23:12:30.417343+0000","last_active":"2024-09-21T23:12:30.416609+0000","last_peered":"2024-09-21T23:12:30.416609+0000","last_clean":"2024-09-21T23:12:30.416609+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:12:30.416609+0000","last_undegraded":"2024-09-21T23:12:30.416609+0000","last_fullsized":"2024-09-21T23:12:30.416609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:43:11.334921+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0097446330000000008,"stat_sum":{"num_bytes":21963219,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":30,"num_read":6639,"num_read_kb":20514,"num_write":473,"num_write_kb":13136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.1","version":"0'0","reported_seq":277,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.404288+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:12:30.404288+0000","last_peered":"2024-09-21T23:12:30.404288+0000","last_clean":"2024-09-21T23:12:30.404288+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:12:30.404288+0000","last_undegraded":"2024-09-21T23:12:30.404288+0000","last_fullsized":"2024-09-21T23:12:30.404288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"149'443","reported_seq":5502,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.408517+0000","last_change":"2024-09-21T23:12:30.412772+0000","last_active":"2024-09-21T23:12:30.408517+0000","last_peered":"2024-09-21T23:12:30.408517+0000","last_clean":"2024-09-21T23:12:30.408517+0000","last_became_active":"2024-09-21T23:12:27.615744+0000","last_became_peered":"2024-09-21T23:12:27.615744+0000","last_unstale":"2024-09-21T23:12:30.408517+0000","last_undegraded":"2024-09-21T23:12:30.408517+0000","last_fullsized":"2024-09-21T23:12:30.408517+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:20.065184+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.017204984999999999,"stat_sum":{"num_bytes":29489687,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":28,"num_read":4756,"num_read_kb":20310,"num_write":488,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.412112+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:12:30.412112+0000","last_peered":"2024-09-21T23:12:30.412112+0000","last_clean":"2024-09-21T23:12:30.412112+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:12:30.412112+0000","last_undegraded":"2024-09-21T23:12:30.412112+0000","last_fullsized":"2024-09-21T23:12:30.412112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"149'545","reported_seq":4212,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.411860+0000","last_change":"2024-09-21T23:12:30.411860+0000","last_active":"2024-09-21T23:12:30.411860+0000","last_peered":"2024-09-21T23:12:30.411860+0000","last_clean":"2024-09-21T23:12:30.411860+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:12:30.411860+0000","last_undegraded":"2024-09-21T23:12:30.411860+0000","last_fullsized":"2024-09-21T23:12:30.411860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:09:19.454925+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014500083,"stat_sum":{"num_bytes":5587875,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":21,"num_read":5245,"num_read_kb":19489,"num_write":628,"num_write_kb":14848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.3","version":"18'1","reported_seq":279,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.402456+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:12:30.402456+0000","last_peered":"2024-09-21T23:12:30.402456+0000","last_clean":"2024-09-21T23:12:30.402456+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:12:30.402456+0000","last_undegraded":"2024-09-21T23:12:30.402456+0000","last_fullsized":"2024-09-21T23:12:30.402456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":376,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.417345+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:12:30.417345+0000","last_peered":"2024-09-21T23:12:30.417345+0000","last_clean":"2024-09-21T23:12:30.417345+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:12:30.417345+0000","last_undegraded":"2024-09-21T23:12:30.417345+0000","last_fullsized":"2024-09-21T23:12:30.417345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"148'477","reported_seq":3405,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.398488+0000","last_change":"2024-09-21T23:12:30.400110+0000","last_active":"2024-09-21T23:12:30.398488+0000","last_peered":"2024-09-21T23:12:30.398488+0000","last_clean":"2024-09-21T23:12:30.398488+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:12:30.398488+0000","last_undegraded":"2024-09-21T23:12:30.398488+0000","last_fullsized":"2024-09-21T23:12:30.398488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:46.809754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027524049999999999,"stat_sum":{"num_bytes":3532001,"num_objects":33,"num_object_clones":2,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":22,"num_read":4462,"num_read_kb":9858,"num_write":473,"num_write_kb":5012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.9","version":"149'482","reported_seq":4052,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.404375+0000","last_change":"2024-09-21T23:12:30.404375+0000","last_active":"2024-09-21T23:12:30.404375+0000","last_peered":"2024-09-21T23:12:30.404375+0000","last_clean":"2024-09-21T23:12:30.404375+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:12:30.404375+0000","last_undegraded":"2024-09-21T23:12:30.404375+0000","last_fullsized":"2024-09-21T23:12:30.404375+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:14.545544+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093147509999999996,"stat_sum":{"num_bytes":21403409,"num_objects":37,"num_object_clones":2,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":23,"num_read":5739,"num_read_kb":19272,"num_write":446,"num_write_kb":5926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.a","version":"149'582","reported_seq":6333,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.418009+0000","last_change":"2024-09-21T23:12:30.418009+0000","last_active":"2024-09-21T23:12:30.418009+0000","last_peered":"2024-09-21T23:12:30.418009+0000","last_clean":"2024-09-21T23:12:30.418009+0000","last_became_active":"2024-09-21T23:12:28.638646+0000","last_became_peered":"2024-09-21T23:12:28.638646+0000","last_unstale":"2024-09-21T23:12:30.418009+0000","last_undegraded":"2024-09-21T23:12:30.418009+0000","last_fullsized":"2024-09-21T23:12:30.418009+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:55:34.119555+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010081031000000001,"stat_sum":{"num_bytes":25781786,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":32,"num_read":6692,"num_read_kb":40119,"num_write":515,"num_write_kb":11929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.b","version":"149'386","reported_seq":3545,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.429475+0000","last_change":"2024-09-21T23:12:30.429943+0000","last_active":"2024-09-21T23:12:30.429475+0000","last_peered":"2024-09-21T23:12:30.429475+0000","last_clean":"2024-09-21T23:12:30.429475+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:12:30.429475+0000","last_undegraded":"2024-09-21T23:12:30.429475+0000","last_fullsized":"2024-09-21T23:12:30.429475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:02.892506+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.032306336999999997,"stat_sum":{"num_bytes":20431022,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":24,"num_read":3515,"num_read_kb":26403,"num_write":338,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.c","version":"149'501","reported_seq":2126,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.431919+0000","last_change":"2024-09-21T23:12:30.433165+0000","last_active":"2024-09-21T23:12:30.431919+0000","last_peered":"2024-09-21T23:12:30.431919+0000","last_clean":"2024-09-21T23:12:30.431919+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:12:30.431919+0000","last_undegraded":"2024-09-21T23:12:30.431919+0000","last_fullsized":"2024-09-21T23:12:30.431919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:43.238901+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.028748961,"stat_sum":{"num_bytes":18132273,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":35,"num_read":2206,"num_read_kb":17676,"num_write":472,"num_write_kb":9186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.d","version":"149'506","reported_seq":5056,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.446359+0000","last_change":"2024-09-21T23:12:30.447089+0000","last_active":"2024-09-21T23:12:30.446359+0000","last_peered":"2024-09-21T23:12:30.446359+0000","last_clean":"2024-09-21T23:12:30.446359+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:12:30.446359+0000","last_undegraded":"2024-09-21T23:12:30.446359+0000","last_fullsized":"2024-09-21T23:12:30.446359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:05:13.332782+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.032638522000000003,"stat_sum":{"num_bytes":51267672,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":36,"num_read":5085,"num_read_kb":28814,"num_write":427,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.e","version":"149'470","reported_seq":4869,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.438310+0000","last_change":"2024-09-21T23:12:30.439292+0000","last_active":"2024-09-21T23:12:30.438310+0000","last_peered":"2024-09-21T23:12:30.438310+0000","last_clean":"2024-09-21T23:12:30.438310+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:12:30.438310+0000","last_undegraded":"2024-09-21T23:12:30.438310+0000","last_fullsized":"2024-09-21T23:12:30.438310+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:32:43.974014+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.026362712999999999,"stat_sum":{"num_bytes":45090041,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":29,"num_read":6691,"num_read_kb":23678,"num_write":457,"num_write_kb":9490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.f","version":"149'541","reported_seq":2406,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.450783+0000","last_change":"2024-09-21T23:12:30.450783+0000","last_active":"2024-09-21T23:12:30.450783+0000","last_peered":"2024-09-21T23:12:30.450783+0000","last_clean":"2024-09-21T23:12:30.450783+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:12:30.450783+0000","last_undegraded":"2024-09-21T23:12:30.450783+0000","last_fullsized":"2024-09-21T23:12:30.450783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:00:43.617256+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.033728045999999998,"stat_sum":{"num_bytes":16920537,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":37,"num_read":2344,"num_read_kb":21660,"num_write":595,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.10","version":"149'619","reported_seq":3912,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.436497+0000","last_change":"2024-09-21T23:12:30.437098+0000","last_active":"2024-09-21T23:12:30.436497+0000","last_peered":"2024-09-21T23:12:30.436497+0000","last_clean":"2024-09-21T23:12:30.436497+0000","last_became_active":"2024-09-21T23:09:55.381124+0000","last_became_peered":"2024-09-21T23:09:55.381124+0000","last_unstale":"2024-09-21T23:12:30.436497+0000","last_undegraded":"2024-09-21T23:12:30.436497+0000","last_fullsized":"2024-09-21T23:12:30.436497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:45:27.654228+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.029322373999999998,"stat_sum":{"num_bytes":49003068,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":26,"num_read":5870,"num_read_kb":37397,"num_write":643,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.11","version":"149'621","reported_seq":6778,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.424729+0000","last_change":"2024-09-21T23:12:30.425145+0000","last_active":"2024-09-21T23:12:30.424729+0000","last_peered":"2024-09-21T23:12:30.424729+0000","last_clean":"2024-09-21T23:12:30.424729+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:12:30.424729+0000","last_undegraded":"2024-09-21T23:12:30.424729+0000","last_fullsized":"2024-09-21T23:12:30.424729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:25.003501+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012422231000000001,"stat_sum":{"num_bytes":59611231,"num_objects":47,"num_object_clones":5,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":32,"num_read":7995,"num_read_kb":42900,"num_write":656,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.12","version":"149'575","reported_seq":6956,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.542204+0000","last_change":"2024-09-21T23:12:30.440535+0000","last_active":"2024-09-21T23:12:30.542204+0000","last_peered":"2024-09-21T23:12:30.542204+0000","last_clean":"2024-09-21T23:12:30.542204+0000","last_became_active":"2024-09-21T23:11:15.382800+0000","last_became_peered":"2024-09-21T23:11:15.382800+0000","last_unstale":"2024-09-21T23:12:30.542204+0000","last_undegraded":"2024-09-21T23:12:30.542204+0000","last_fullsized":"2024-09-21T23:12:30.542204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:00.493384+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.044612595999999997,"stat_sum":{"num_bytes":20971583,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":7899,"num_read_kb":39496,"num_write":632,"num_write_kb":21744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.13","version":"149'393","reported_seq":2410,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T23:12:30.442561+0000","last_change":"2024-09-21T23:12:30.443448+0000","last_active":"2024-09-21T23:12:30.442561+0000","last_peered":"2024-09-21T23:12:30.442561+0000","last_clean":"2024-09-21T23:12:30.442561+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:12:30.442561+0000","last_undegraded":"2024-09-21T23:12:30.442561+0000","last_fullsized":"2024-09-21T23:12:30.442561+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:58:48.133887+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.029852179,"stat_sum":{"num_bytes":14696228,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":29,"num_read":1454,"num_read_kb":14144,"num_write":372,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":968343671,"num_objects":1410,"num_object_clones":203,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":880,"num_read":179581,"num_read_kb":835151,"num_write":21283,"num_write_kb":399656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4499,"num_bytes_recovered":251738539,"num_keys_recovered":713,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":357224448,"data_stored":381823947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":19309,"ondisk_log_size":19309,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123884,"kb_used_data":66320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94247956,"statfs":{"total":96636764160,"available":96509906944,"internally_reserved":0,"allocated":67911680,"data_stored":68928641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64436,"kb_used_data":24924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94307404,"statfs":{"total":96636764160,"available":96570781696,"internally_reserved":0,"allocated":25522176,"data_stored":25047928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81712,"kb_used_data":50524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94290128,"statfs":{"total":96636764160,"available":96553091072,"internally_reserved":0,"allocated":51736576,"data_stored":58525254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743443,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99264,"kb_used_data":62272,"kb_used_omap":15,"kb_used_meta":36912,"kb_avail":94272576,"statfs":{"total":96636764160,"available":96535117824,"internally_reserved":0,"allocated":63766528,"data_stored":65919678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":37798850},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124788,"kb_used_data":53316,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94247052,"statfs":{"total":96636764160,"available":96508981248,"internally_reserved":0,"allocated":54595584,"data_stored":56814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63412,"kb_used_data":19740,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94308428,"statfs":{"total":96636764160,"available":96571830272,"internally_reserved":0,"allocated":20213760,"data_stored":20970002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81500,"kb_used_data":32220,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94290340,"statfs":{"total":96636764160,"available":96553308160,"internally_reserved":0,"allocated":32993280,"data_stored":34125646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122908,"kb_used_data":69512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94248932,"statfs":{"total":96636764160,"available":96510906368,"internally_reserved":0,"allocated":71180288,"data_stored":75333049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65449984,"data_stored":70409921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28504064,"data_stored":30441474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16830464,"data_stored":18439040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":51691520,"data_stored":54835458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":59912192,"data_stored":62869983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46661632,"data_stored":54253453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22646784,"data_stored":23101367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65527808,"data_stored":67473251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:12:31.615 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:12:31.615 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:31.605+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:31.616 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:31.616 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T23:12:31.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:12:31.712 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:31.705+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:31.780 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:31.780 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:31.780 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:31.780 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:31.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:31.813 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:31.809+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:31.914 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:31.909+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:31.917 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:31.917 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:31.917 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:31.917 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:31.917 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:31.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "epoch": 150, 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "fsid": "9ce7cfc1-a7d6-426f-bf44-a308fb883b8f", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "created": "2024-09-21T23:09:25.662940+0000", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "modified": "2024-09-21T23:12:31.391501+0000", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "last_up_change": "2024-09-21T23:11:01.346212+0000", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "last_in_change": "2024-09-21T23:11:10.384786+0000", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_num": 5801984, 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_set": [ 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "noscrub", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "pglog_hardlimit", 2024-09-21T23:12:31.961 INFO:teuthology.orchestra.run.smithi150.stdout: "purged_snapdirs", 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_deletes", 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "sortbitwise" 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_version": 5, 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_snapshot": "", 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_max": 3, 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd": 8, 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "min_compat_client": "luminous", 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "require_osd_release": "squid", 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_crimson": false, 2024-09-21T23:12:31.962 INFO:teuthology.orchestra.run.smithi150.stdout: "pools": [ 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 1, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": ".mgr", 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:40.569134+0000", 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool", 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:12:31.963 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 1, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 1, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 1, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 1, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 1, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.964 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "16", 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 0, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 0, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:12:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "options": { 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_max": 32, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_min": 1 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr": {} 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.967 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 8, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 8, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 0.25, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 2, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 2, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 2, 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": "rbd", 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:44.626156+0000", 2024-09-21T23:12:31.968 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 8193, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 8, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 8, 2024-09-21T23:12:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 8, 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 8, 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 8, 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "20", 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:12:31.970 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 2, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 20, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:12:31.971 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:12:31.972 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "options": {}, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd": {} 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:12:31.973 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 1.9900000095367432, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 1.9900000095367432, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 1.75, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 1.75, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 3, 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": "pool_client.0", 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "create_time": "2024-09-21T23:09:53.888601+0000", 2024-09-21T23:12:31.974 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 8193, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type": 1, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 2, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": 1, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_rule": 0, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "is_stretch_pool": false, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "object_hash": 2, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": 32, 2024-09-21T23:12:31.975 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num": 32, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_placement_num_target": 32, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_target": 32, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num_pending": 32, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "last_pg_merge_meta": { 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "source_pgid": "0.0", 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "ready_epoch": 0, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_started": 0, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "last_epoch_clean": 0, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "source_version": "0'0", 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "target_version": "0'0" 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "last_change": "150", 2024-09-21T23:12:31.976 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend": "0", 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "auid": 0, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_seq": 54, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "snap_epoch": 150, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_snaps": [], 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps": "[]", 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_bytes": 0, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "quota_max_objects": 0, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "tiers": [], 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "tier_of": -1, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "read_tier": -1, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "write_tier": -1, 2024-09-21T23:12:31.977 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_mode": "none", 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_bytes": 0, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "target_max_objects": 0, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_flush_age": 0, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "cache_min_evict_age": 0, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profile": "", 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_params": { 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "none" 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_period": 0, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_count": 0, 2024-09-21T23:12:31.978 INFO:teuthology.orchestra.run.smithi150.stdout: "use_gmt_hitset": true, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "grade_table": [], 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "stripe_width": 0, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "expected_num_objects": 0, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "fast_read": false, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "options": {}, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "application_metadata": { 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd": {} 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "read_balance": { 2024-09-21T23:12:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: "score_type": "Fair distribution", 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "score_acting": 1.5, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "score_stable": 1.5, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "optimal_score": 1, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_acting": 1.5, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "raw_score_stable": 1.5, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity": 1, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "osds": [ 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 0, 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d", 2024-09-21T23:12:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 145, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6816", 2024-09-21T23:12:31.981 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6817", 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6818", 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6819", 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6822", 2024-09-21T23:12:31.983 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6823", 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6820", 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6821", 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 198356244 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6817/198356244", 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6819/198356244", 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6823/198356244", 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6821/198356244", 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:31.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 1, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "5fb69564-3b15-470e-b45f-f70594751344", 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 145, 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6824", 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6825", 2024-09-21T23:12:31.987 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6826", 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.988 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6827", 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6830", 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6831", 2024-09-21T23:12:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6828", 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6829", 2024-09-21T23:12:31.990 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 725319294 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6825/725319294", 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6827/725319294", 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6831/725319294", 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6829/725319294", 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 2, 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "cc3e3e88-a324-4e3d-8b40-0d8c93da1400", 2024-09-21T23:12:31.991 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 144, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6800", 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6801", 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6802", 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6803", 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6806", 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6807", 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6804", 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.995 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6805", 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1956153436 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6801/1956153436", 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6803/1956153436", 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6807/1956153436", 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6805/1956153436", 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 3, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "933fe042-8aed-4795-918c-572cf81999a7", 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 82, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:31.997 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6808", 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6809", 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6810", 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6811", 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:31.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6814", 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6815", 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.000 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6812", 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6813", 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3049870482 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.150:6809/3049870482", 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.150:6811/3049870482", 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.150:6815/3049870482", 2024-09-21T23:12:32.001 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.150:6813/3049870482", 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 4, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "af826961-2dad-4702-a904-991e2f5f57c8", 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 13, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 32, 2024-09-21T23:12:32.002 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 69, 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 147, 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 33, 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6800", 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6801", 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.003 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6802", 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6803", 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6806", 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6807", 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6804", 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6805", 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3379790213 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6801/3379790213", 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6803/3379790213", 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6807/3379790213", 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6805/3379790213", 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 5, 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "c4f1908b-0885-408e-873e-4d2aa9b9c506", 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:32.007 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 81, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6826", 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.008 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6827", 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6828", 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6829", 2024-09-21T23:12:32.009 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6832", 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.010 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6833", 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6830", 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6831", 2024-09-21T23:12:32.011 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 120048378 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6827/120048378", 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6829/120048378", 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6833/120048378", 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6831/120048378", 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 6, 2024-09-21T23:12:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "08cef493-ce38-4b52-b5f2-23e7694953c0", 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 146, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6818", 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6819", 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.014 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6820", 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6821", 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6824", 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6825", 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6822", 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6823", 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 787648514 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6819/787648514", 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6821/787648514", 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6825/787648514", 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6823/787648514", 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:32.017 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 7, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "uuid": "26d2142e-419c-4da3-9c28-ac442b1b66fb", 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "up": 1, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "in": 1, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "weight": 1, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_affinity": 1, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_begin": 0, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "last_clean_end": 0, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "up_from": 13, 2024-09-21T23:12:32.018 INFO:teuthology.orchestra.run.smithi150.stdout: "up_thru": 146, 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "down_at": 0, 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "lost_at": 0, 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addrs": { 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6808", 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6809", 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addrs": { 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6810", 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6811", 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addrs": { 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6814", 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6815", 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.021 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addrs": { 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6812", 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.175:6813", 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1424462696 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "public_addr": "172.21.15.175:6809/1424462696", 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_addr": "172.21.15.175:6811/1424462696", 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_back_addr": "172.21.15.175:6815/1424462696", 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "heartbeat_front_addr": "172.21.15.175:6813/1424462696", 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "state": [ 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "exists", 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "up" 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_xinfo": [ 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 0, 2024-09-21T23:12:32.023 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:36.004700+0000", 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 1, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.024 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.958497+0000", 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 2, 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.988302+0000", 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.025 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 3, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.993971+0000", 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 4, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.026 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.945648+0000", 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 33 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 5, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:36.366302+0000", 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:12:32.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 6, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.852613+0000", 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "osd": 7, 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "down_stamp": "0.000000", 2024-09-21T23:12:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_probability": 0, 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "laggy_interval": 0, 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "features": 4540701547738038271, 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "old_weight": 0, 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "last_purged_snaps_scrub": "2024-09-21T23:09:35.713697+0000", 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "dead_epoch": 0 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap": [], 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap_items": [ 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "pgid": "3.18", 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "mappings": [ 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "from": 3, 2024-09-21T23:12:32.029 INFO:teuthology.orchestra.run.smithi150.stdout: "to": 2 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_upmap_primaries": [], 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_temp": [], 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "primary_temp": [], 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklist": {}, 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "range_blocklist": {}, 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "erasure_code_profiles": { 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "default": { 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-failure-domain": "osd", 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "k": "2", 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "m": "1", 2024-09-21T23:12:32.030 INFO:teuthology.orchestra.run.smithi150.stdout: "plugin": "jerasure", 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "technique": "reed_sol_van" 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "removed_snaps_queue": [ 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 3, 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "snaps": [ 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "begin": 25, 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "length": 5 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: "new_removed_snaps": [ 2024-09-21T23:12:32.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "pool": 3, 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "snaps": [ 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "begin": 25, 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "length": 1 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "new_purged_snaps": [], 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_node_flags": {}, 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "device_class_flags": {}, 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode": { 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode_enabled": false, 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_bucket_count": 0, 2024-09-21T23:12:32.032 INFO:teuthology.orchestra.run.smithi150.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:12:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:12:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: "stretch_mode_bucket": 0 2024-09-21T23:12:32.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.033 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:32.034 INFO:tasks.thrashosds.thrasher:j is {'epoch': 150, 'fsid': '9ce7cfc1-a7d6-426f-bf44-a308fb883b8f', 'created': '2024-09-21T23:09:25.662940+0000', 'modified': '2024-09-21T23:12:31.391501+0000', 'last_up_change': '2024-09-21T23:11:01.346212+0000', 'last_in_change': '2024-09-21T23:11:10.384786+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:09:40.569134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:09:44.626156+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T23:09:53.888601+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 54, 'snap_epoch': 150, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 145, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 198356244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 198356244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 198356244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 198356244}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 198356244}]}, 'public_addr': '172.21.15.150:6817/198356244', 'cluster_addr': '172.21.15.150:6819/198356244', 'heartbeat_back_addr': '172.21.15.150:6823/198356244', 'heartbeat_front_addr': '172.21.15.150:6821/198356244', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '5fb69564-3b15-470e-b45f-f70594751344', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 145, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 725319294}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 725319294}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 725319294}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 725319294}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 725319294}]}, 'public_addr': '172.21.15.150:6825/725319294', 'cluster_addr': '172.21.15.150:6827/725319294', 'heartbeat_back_addr': '172.21.15.150:6831/725319294', 'heartbeat_front_addr': '172.21.15.150:6829/725319294', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'cc3e3e88-a324-4e3d-8b40-0d8c93da1400', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 144, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 1956153436}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 1956153436}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 1956153436}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 1956153436}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 1956153436}]}, 'public_addr': '172.21.15.150:6801/1956153436', 'cluster_addr': '172.21.15.150:6803/1956153436', 'heartbeat_back_addr': '172.21.15.150:6807/1956153436', 'heartbeat_front_addr': '172.21.15.150:6805/1956153436', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '933fe042-8aed-4795-918c-572cf81999a7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 3049870482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 3049870482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 3049870482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 3049870482}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 3049870482}]}, 'public_addr': '172.21.15.150:6809/3049870482', 'cluster_addr': '172.21.15.150:6811/3049870482', 'heartbeat_back_addr': '172.21.15.150:6815/3049870482', 'heartbeat_front_addr': '172.21.15.150:6813/3049870482', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'af826961-2dad-4702-a904-991e2f5f57c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 32, 'up_from': 69, 'up_thru': 147, 'down_at': 33, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 3379790213}, {'type': 'v1', 'addr': '172.21.15.175:6801', 'nonce': 3379790213}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 3379790213}, {'type': 'v1', 'addr': '172.21.15.175:6803', 'nonce': 3379790213}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 3379790213}, {'type': 'v1', 'addr': '172.21.15.175:6807', 'nonce': 3379790213}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 3379790213}, {'type': 'v1', 'addr': '172.21.15.175:6805', 'nonce': 3379790213}]}, 'public_addr': '172.21.15.175:6801/3379790213', 'cluster_addr': '172.21.15.175:6803/3379790213', 'heartbeat_back_addr': '172.21.15.175:6807/3379790213', 'heartbeat_front_addr': '172.21.15.175:6805/3379790213', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c4f1908b-0885-408e-873e-4d2aa9b9c506', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6826', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6827', 'nonce': 120048378}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6828', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6829', 'nonce': 120048378}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 120048378}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6830', 'nonce': 120048378}, {'type': 'v1', 'addr': '172.21.15.175:6831', 'nonce': 120048378}]}, 'public_addr': '172.21.15.175:6827/120048378', 'cluster_addr': '172.21.15.175:6829/120048378', 'heartbeat_back_addr': '172.21.15.175:6833/120048378', 'heartbeat_front_addr': '172.21.15.175:6831/120048378', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '08cef493-ce38-4b52-b5f2-23e7694953c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6818', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6819', 'nonce': 787648514}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6820', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6821', 'nonce': 787648514}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6824', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6825', 'nonce': 787648514}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6822', 'nonce': 787648514}, {'type': 'v1', 'addr': '172.21.15.175:6823', 'nonce': 787648514}]}, 'public_addr': '172.21.15.175:6819/787648514', 'cluster_addr': '172.21.15.175:6821/787648514', 'heartbeat_back_addr': '172.21.15.175:6825/787648514', 'heartbeat_front_addr': '172.21.15.175:6823/787648514', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '26d2142e-419c-4da3-9c28-ac442b1b66fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6809', 'nonce': 1424462696}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6811', 'nonce': 1424462696}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6815', 'nonce': 1424462696}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 1424462696}, {'type': 'v1', 'addr': '172.21.15.175:6813', 'nonce': 1424462696}]}, 'public_addr': '172.21.15.175:6809/1424462696', 'cluster_addr': '172.21.15.175:6811/1424462696', 'heartbeat_back_addr': '172.21.15.175:6815/1424462696', 'heartbeat_front_addr': '172.21.15.175:6813/1424462696', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:36.004700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.958497+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.988302+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.993971+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.945648+0000', 'dead_epoch': 33}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:36.366302+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.852613+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:09:35.713697+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.18', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:12:32.035 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.18 2024-09-21T23:12:32.035 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.18 2024-09-21T23:12:32.035 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:32.032+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 90.661501657000002, 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.055 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:32.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191536787999993, 2024-09-21T23:12:32.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191513633, 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.063 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:12:32.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:12:32.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191491416000005, 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.067 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:12:32.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:12:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191470804999994, 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.071 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:32.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:12:32.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191451181000005, 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:32.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:12:32.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:12:32.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191430244000003, 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.079 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:12:32.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191408280999994, 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:12:32.083 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.087 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191388857000007, 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.091 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191369715999997, 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191343266000004, 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.099 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191323745999995, 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:12:32.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:12:32.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191300940000005, 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:12:32.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.107 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191280468000002, 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:32.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:12:32.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:12:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.111 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191204702999997, 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:12:32.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.191181030999999, 2024-09-21T23:12:32.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.186265860000006, 2024-09-21T23:12:32.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.186241133999999, 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:12:32.123 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:12:32.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.186216559000002, 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:12:32.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:12:32.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123558 3.9 3:93c04306:::rbd_data.11495bdc1465.0000000000000026:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 5.3052670930000003, 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88255402500000002, 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123558 2024-09-21T23:12:32.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746441+0000", 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0530000000000001e-06 2024-09-21T23:12:32.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746450+0000", 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6889999999999993e-06 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746471+0000", 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1005999999999998e-05 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746485+0000", 2024-09-21T23:12:32.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746627+0000", 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014169399999999999 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746638+0000", 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1311000000000001e-05 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621095+0000", 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87445676999999999 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621273+0000", 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017780399999999999 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2", 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621358+0000", 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5068999999999995e-05 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.623621+0000", 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022634690000000002 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628925+0000", 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053032009999999996 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628961+0000", 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5936000000000003e-05 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628993+0000", 2024-09-21T23:12:32.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2274000000000003e-05 2024-09-21T23:12:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:32.137 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:32.137 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:32.138 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:32.139 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:32.137+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:32.240 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:32.233+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:32.301 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:32.301 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:32.315 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:32.360 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:32.357+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:32.441 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:32.440+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:32.444 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:12:32.487 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:32.487 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:32.487 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:32.487 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:32.488 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:32.503 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:32.545 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:32.541+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:32.642 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:32.640+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:32.671 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:32.671 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:32.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:32.671 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.06663650900001, 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:12:32.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:12:32.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.066589842, 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.066515776, 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.06645305000001, 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:12:32.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:12:32.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.06640109999999, 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:12:32.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:12:32.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.06618801400001, 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.692 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:12:32.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:12:32.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.064303989, 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.06425593900001, 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.700 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:12:32.703 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.06420763200001, 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:12:32.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:12:32.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.707 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.064160417, 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.708 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.06409618799999, 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:32.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:12:32.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.064044278, 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:12:32.716 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:32.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:12:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:12:32.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.059441984, 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.720 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:32.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:12:32.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:12:32.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.059387169, 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.05933944500001, 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.059272556, 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:12:32.732 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.05572727800001, 2024-09-21T23:12:32.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.055680537, 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:12:32.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.744 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.055634015, 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:12:32.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 85.260937837, 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:12:32.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:12:32.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.753 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.754 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.754 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:32.754 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:32.754 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:32.755 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:32.756 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:32.749+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:32.846 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:32.897 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:32.893+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:32.922 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:32.923 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:32.923 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:32.923 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:32.937 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:32.958 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:32.956+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:32.993 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:32.993 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:33.012 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:33.028 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:33.045 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:33.045 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:33.058 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:33.053+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:33.066 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:33.066 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:33.086 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:33.086 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:33.099 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:33.099 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:33.107 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:33.107 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:33.113 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:33.128 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:33.129 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:33.158 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:33.156+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:33.240 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:33.240 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:33.252 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:33.301 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:33.297+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:33.359 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:33.353+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:33.367 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:33.367 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:33.381 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:33.411 INFO:teuthology.orchestra.run.smithi150.stderr:clear 3.18 pg_upmap_items mapping 2024-09-21T23:12:33.411 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:12:33.460 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:33.456+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:33.529 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:33.529 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:33.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:33.592 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:33.589+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:33.661 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:33.660+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:33.665 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:33.665 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:33.679 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:33.761 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:33.760+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:33.841 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:33.841 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:33.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:33.862 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:33.860+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:33.963 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:33.957+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:33.978 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:33.978 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:33.992 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:34.081 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:34.077+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:34.156 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:34.156 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:34.164 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:34.160+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:34.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:34.265 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:34.264+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:34.304 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:34.304 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:34.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:34.367 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:34.365+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:34.468 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:34.461+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:34.485 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:34.485 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:34.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:34.569 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:34.564+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:34.622 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:34.622 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:34.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:34.677 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:34.676+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:34.771 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:34.765+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:34.795 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:34.795 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:34.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:34.871 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:34.868+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:34.925 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:34.925 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:34.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:34.972 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:34.965+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:35.072 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:35.072+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:35.108 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:35.108 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:35.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:35.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:35.193 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:35.189+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:35.274 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:35.272+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:35.292 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:35.292 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:35.300 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:35.301 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:35.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:35.317 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:35.317 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:35.334 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:35.334 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:35.351 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:35.351 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:35.370 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:35.370 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:35.375 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:35.369+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:35.389 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:35.389 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:35.410 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:35.411 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:35.431 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:35.431 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:35.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:12:35.476 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:35.472+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:35.489 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:35.489 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:35.490 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:35.490 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:35.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:35.593 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:35.592+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:35.675 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:35.675 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.48818447900001, 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.676 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.48808139299999, 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.680 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:12:35.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.48801404400001, 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:35.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:12:35.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:12:35.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.41282710799999, 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:12:35.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:12:35.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.41277134000001, 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.692 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.41273980099999, 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.412707052, 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:12:35.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.41268015899999, 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.41265488600001, 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:12:35.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:12:35.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.41262375900001, 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.713 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:35.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.411980403, 2024-09-21T23:12:35.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:12:35.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:12:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.406772993, 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:12:35.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:12:35.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:12:35.725 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.40674870699999, 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.726 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.40673104000001, 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.730 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.734 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.406707455, 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:35.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.40669144, 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.739 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:12:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:12:35.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.40667760900001, 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.743 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:12:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:12:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.40664624799999, 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:12:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:12:35.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 148.40657782599999, 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:12:35.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:12:35.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123547 3.13 3:c9e2c0b1:::rbd_data.11495bdc1465.000000000000000d:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.9298903369999998, 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88065156200000005, 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:35.756 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123547 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742119+0000", 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0244e-05 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742127+0000", 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6100000000000006e-06 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742154+0000", 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6730999999999999e-05 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742165+0000", 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1419e-05 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742241+0000", 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5562999999999995e-05 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742251+0000", 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0017000000000001e-05 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619014+0000", 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87676288899999999 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619157+0000", 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143549 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 4", 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619280+0000", 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012234100000000001 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621009+0000", 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001729511 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622618+0000", 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001609262 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622687+0000", 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8813000000000007e-05 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:35.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622740+0000", 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2612999999999999e-05 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:35.762 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:35.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:35.764 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:35.757+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:35.864 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:35.861+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:35.932 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:35.932 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:35.932 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:35.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:35.965 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:35.961+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:36.066 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:36.061+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:36.068 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:36.068 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:36.068 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:36.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:36.167 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:36.161+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.814442546999999, 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.208 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:12:36.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344477678000004, 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:12:36.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:12:36.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344454522999996, 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344432306000002, 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344411695000005, 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344392071000001, 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:12:36.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:12:36.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:12:36.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344371133999999, 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:12:36.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344349171000005, 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344329747000003, 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:12:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.245 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344310605999993, 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.249 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344284156000001, 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.250 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344264636000005, 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.254 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.258 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344241830000001, 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:12:36.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:12:36.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344221357999999, 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:12:36.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344145592999993, 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:12:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.344121920999996, 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:12:36.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.339206750000002, 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:12:36.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:12:36.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.339182023999996, 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 74.339157448999998, 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123558 3.9 3:93c04306:::rbd_data.11495bdc1465.0000000000000026:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.4582079829999994, 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88255402500000002, 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123558 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.287 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746441+0000", 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0530000000000001e-06 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746450+0000", 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6889999999999993e-06 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746471+0000", 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1005999999999998e-05 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746485+0000", 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746627+0000", 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014169399999999999 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746638+0000", 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1311000000000001e-05 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621095+0000", 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87445676999999999 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621273+0000", 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017780399999999999 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2", 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621358+0000", 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5068999999999995e-05 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.623621+0000", 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022634690000000002 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628925+0000", 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053032009999999996 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628961+0000", 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5936000000000003e-05 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628993+0000", 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2274000000000003e-05 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:36.293 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:36.293 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:36.294 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:36.292+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:36.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:36.394 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:36.392+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:36.419 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:12:36.431 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:36.432 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:36.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:36.496 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:36.492+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:36.573 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:36.573 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:36.573 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:36.587 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:36.596 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:36.596+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:36.697 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:36.693+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:36.706 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:36.706 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:36.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.100520057, 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.707 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.10047339, 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.711 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:12:36.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:12:36.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.10039932399999, 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:12:36.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:12:36.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:12:36.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:12:36.718 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.100336598, 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.719 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:12:36.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.100284648, 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:12:36.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:12:36.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.100071562, 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:12:36.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:12:36.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.730 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.098187537, 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:12:36.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:12:36.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:12:36.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:12:36.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.098139487, 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.735 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:12:36.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.09809118, 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.739 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.098043965, 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.743 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:12:36.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.746 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.097979736, 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:12:36.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:12:36.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:12:36.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.097927826, 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.751 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:12:36.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:12:36.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:12:36.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.09332553199999, 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.755 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:12:36.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:12:36.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:12:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.093270717, 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.759 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.093222993, 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.763 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.093156104, 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.767 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.089610826, 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.771 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.08956408500001, 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:12:36.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:12:36.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.089517563, 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.779 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:12:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 89.294821385000006, 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:36.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:12:36.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:12:36.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.787 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.788 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.788 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:36.788 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:36.788 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:36.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:36.833 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:36.829+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:36.898 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:36.896+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:36.957 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:36.957 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:36.957 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:36.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:36.999 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:36.996+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:37.092 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:37.092 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:37.092 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:37.092 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:37.092 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:37.100 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:37.097+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:37.106 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:37.202 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:37.200+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.903443319000004, 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.274 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:12:37.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:12:37.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612181144999994, 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:12:37.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:12:37.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612151396000002, 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.283 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:12:37.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.286 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612130288000003, 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612095406999998, 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:12:37.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612072998000002, 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612051542000003, 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.299 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612020876000003, 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:12:37.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.612001140000004, 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:12:37.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.611981155999999, 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:12:37.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:12:37.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.314 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.611962387000005, 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:37.318 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.611943213999993, 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:37.322 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.611924294000005, 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.598815016000003, 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:12:37.326 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:12:37.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.598789150000002, 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:12:37.330 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:12:37.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.598754526999997, 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.598727871999998, 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.598704428000005, 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:12:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:12:37.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.598684220999999, 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:12:37.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:12:37.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.598645040999997, 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:37.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:37.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:37.353 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:37.354 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:37.355 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:37.355 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:37.349+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:37.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:37.456 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:37.452+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:37.531 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:37.531 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:37.531 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:37.531 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:37.546 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:37.593 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:37.589+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:37.597 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:37.597 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:37.614 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:37.614 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:37.632 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:37.632 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:37.648 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:37.649 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:37.657 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:37.656+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:37.661 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:37.661 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:37.668 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:37.669 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:37.675 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:37.689 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:37.690 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:37.711 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:37.711 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:37.732 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:37.732 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:37.797 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:37.793+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:37.804 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:37.804 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:37.819 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:37.865 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:37.861+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:37.960 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:37.953+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:37.986 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:37.986 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:38.000 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:38.060 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:38.060+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:38.162 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:38.160+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:38.187 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:38.262 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:38.257+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:38.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:38.369 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:38.368+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:38.436 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:38.437 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:38.437 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:12:38.437 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-09-21T23:12:38.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-09-21T23:12:38.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:12:38.455 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:38.455 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:38.477 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:38.485 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:38.481+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:38.564 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:38.557+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:38.648 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:38.648 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:38.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:38.665 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:38.661+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:38.767 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:38.761+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:38.785 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:38.785 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:38.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:38.867 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:38.864+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:38.921 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:38.921 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:38.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:38.969 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:38.965+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:39.070 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:39.069+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:39.101 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:39.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:39.170 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:39.168+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:39.241 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:39.241 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:39.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:39.271 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:39.265+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:39.372 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:39.368+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:39.416 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:39.416 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:39.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:39.465 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.0 primary-affinity to 0 (802) 2024-09-21T23:12:39.465 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:12:39.473 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:39.472+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:39.573 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:39.572+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:39.600 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:39.600 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:39.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:39.674 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:39.669+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:39.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:39.775 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:39.772+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:39.777 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:39.778 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:39.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:39.875 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:39.869+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:39.904 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:39.904 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:39.920 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:39.920 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:39.936 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:39.937 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:39.953 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:39.954 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:39.957 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:39.957 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:39.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:39.972 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:39.972 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:39.976 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:39.969+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:40.077 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:40.073+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:40.086 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:40.087 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:40.087 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:40.087 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:40.087 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:40.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:40.178 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:40.176+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:40.194 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:40.194 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:40.222 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:40.222 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.834156707999995, 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.228 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:12:40.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:12:40.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.232 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364191839, 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:40.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:12:40.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:40.236 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364168684000006, 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364146466999998, 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:12:40.240 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.244 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364125856000001, 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.245 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:12:40.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364106231999997, 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.249 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364085294999995, 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:12:40.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364063332000001, 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.257 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:40.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:12:40.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364043907999999, 2024-09-21T23:12:40.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:12:40.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:40.265 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.364024767000004, 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.363998316999997, 2024-09-21T23:12:40.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:12:40.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.273 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.363978797000001, 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.363955990999997, 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.278 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:12:40.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.363935518999995, 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.282 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.363859754000003, 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.363836082000006, 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.358920910999998, 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.294 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:12:40.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.358896185000006, 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.298 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 78.358871609999994, 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.302 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:12:40.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123558 3.9 3:93c04306:::rbd_data.11495bdc1465.0000000000000026:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 13.477922144000001, 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88255402500000002, 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:40.306 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123558 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746441+0000", 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0530000000000001e-06 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746450+0000", 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6889999999999993e-06 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746471+0000", 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1005999999999998e-05 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746485+0000", 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746627+0000", 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014169399999999999 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746638+0000", 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1311000000000001e-05 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:40.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621095+0000", 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87445676999999999 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621273+0000", 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017780399999999999 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2", 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621358+0000", 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5068999999999995e-05 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:40.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.623621+0000", 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022634690000000002 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628925+0000", 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053032009999999996 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628961+0000", 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5936000000000003e-05 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628993+0000", 2024-09-21T23:12:40.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2274000000000003e-05 2024-09-21T23:12:40.312 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.312 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.312 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.312 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:40.312 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:40.312 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:40.312 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:40.313 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:40.312+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:40.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:40.314 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:40.414 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:40.409+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:40.497 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:40.497 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:40.497 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:40.512 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:40.561 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:40.557+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:40.615 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:40.609+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:40.688 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:40.688 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:40.688 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:40.702 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:40.749 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:40.745+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:40.816 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:40.813+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:40.877 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.271092884, 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.27104621700001, 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:12:40.881 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:12:40.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:12:40.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:12:40.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:12:40.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.270972151, 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:12:40.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.270909425, 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:12:40.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:12:40.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:12:40.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:12:40.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.270857475, 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.895 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.270644389, 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.268760364, 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.903 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:12:40.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:12:40.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.268712314, 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:12:40.907 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:12:40.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.268664007, 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:40.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:12:40.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:12:40.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.268616792, 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:40.919 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.268552563, 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:40.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:12:40.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.923 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.268500653, 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.924 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:12:40.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:12:40.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:12:40.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.263898359, 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.928 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:12:40.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.263843544, 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:12:40.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:12:40.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.26379582, 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.263728931, 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.260183653, 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:12:40.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:12:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.947 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.26013691199999, 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:40.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:12:40.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:40.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.26009039, 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.465394212000007, 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:40.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:40.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:12:40.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:12:40.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:12:40.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:12:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:40.960 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:40.960 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:40.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:40.961 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:40.957+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:41.061 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:41.057+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:41.132 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:41.132 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:41.132 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:41.132 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:41.146 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:41.162 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:41.160+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:41.264 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:41.260+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:41.312 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:41.313 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:41.313 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:41.313 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:41.313 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:41.327 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:41.365 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:41.364+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:41.454 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:41.454 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:41.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:41.454 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:41.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.454 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 99.083609863000007, 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:41.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:12:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:12:41.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:12:41.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792347688999996, 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792317940000004, 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792296832000005, 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:12:41.468 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:41.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:12:41.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:12:41.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792261951, 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792239542000004, 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:12:41.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:12:41.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792218086000005, 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:12:41.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:12:41.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:12:41.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:12:41.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792187420000005, 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792167684000006, 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.488 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:12:41.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:12:41.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.491 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792147700000001, 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.492 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:12:41.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:12:41.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.495 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792128930999993, 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.496 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:12:41.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792109757999995, 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:12:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:12:41.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.792090838000007, 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:12:41.504 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:12:41.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.508 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.778981560000005, 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:12:41.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:12:41.512 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.778955694000004, 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.513 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.778921070999999, 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.517 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.778894416, 2024-09-21T23:12:41.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:12:41.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.778870971999993, 2024-09-21T23:12:41.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:12:41.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:41.529 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.778850765000001, 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:41.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:12:41.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 78.778811585, 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:12:41.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:41.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:41.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:41.538 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:41.539 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:12:41.540 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:41.540 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:41.533+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:41.641 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:41.640+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:41.719 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:41.720 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:41.736 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:41.781 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:41.777+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:41.842 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:41.837+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:41.854 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:41.854 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:41.868 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:41.943 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:41.940+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:42.031 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:42.031 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:42.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:42.031 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:42.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.031 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.031 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.53675360299999, 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:42.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.035 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.536700165, 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:42.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.53664919000001, 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:42.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.536598259, 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.044 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:42.048 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.536542948, 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:42.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.052 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.536474033, 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:42.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.531768539, 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:42.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.045615028, 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.062 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:42.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:42.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:42.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.04438710700001, 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:42.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:42.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.044301312, 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:42.070 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:42.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:42.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.074 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.043659717, 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.043584843, 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.079 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:42.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:42.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.043524065, 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:42.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:42.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 118.043487747, 2024-09-21T23:12:42.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:42.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:42.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 113.239025953, 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.092 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:42.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:42.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 102.72248778700001, 2024-09-21T23:12:42.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:42.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:42.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.612579705999998, 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:42.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.612548136999997, 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.612512133999999, 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.485728827000003, 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:42.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:42.124 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:42.125 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:42.126 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:42.124+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:42.227 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:42.224+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:42.251 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:42.252 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:42.252 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:42.252 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:42.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:42.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:42.361 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:42.357+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:42.383 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:42.384 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:42.399 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:42.428 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:42.424+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:42.453 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:42.454 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:42.471 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:42.471 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:42.488 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:12:42.490 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:42.490 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:42.510 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:42.532 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:42.532 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:42.532 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:42.529+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:42.556 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:42.556 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:42.568 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:42.568 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:42.579 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:42.579 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:42.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:42.601 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:42.601 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:42.633 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:42.632+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:42.723 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:42.724 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:42.730 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:42.725+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:42.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:42.831 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:42.825+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:42.906 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:42.906 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:42.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:42.931 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:42.925+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:43.032 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:43.025+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:43.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:43.133 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:43.129+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:43.180 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:43.180 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:43.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:43.234 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:43.229+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:43.317 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:43.317 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:43.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:43.377 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:43.376+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:43.435 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:43.432+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:43.463 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:43.536 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:43.529+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:43.627 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:43.627 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:43.637 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:43.633+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:43.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:43.737 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:43.733+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:43.766 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:43.766 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:43.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:43.838 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:43.833+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:43.904 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:43.905 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:43.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:43.939 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:43.933+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:44.040 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:44.036+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:44.041 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:44.041 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:44.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:44.140 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:44.137+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:44.224 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:44.224 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:44.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:44.241 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:44.237+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:44.341 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:44.340+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:44.355 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:44.355 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:44.371 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:44.442 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:44.437+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:44.497 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:44.497 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:44.498 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T23:12:44.498 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:12:44.498 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:12:44.498 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T23:12:44.511 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:44.511 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:44.513 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:12:44.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:44.577 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:44.573+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:44.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:44.638 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:44.638 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "" 2024-09-21T23:12:44.638 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:44.644 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:44.637+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:44.710 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:44.710 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:44.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:44.744 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:44.744+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:44.780 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:44.780 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:44.798 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:44.799 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:44.833 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:44.833 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:44.845 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:44.844+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:44.853 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:44.853 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40166 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 2957312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.275654835, 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093513182, 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.881 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40166 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602963+0000", 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602964+0000", 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8599999999999997e-07 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602969+0000", 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0429999999999998e-06 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602984+0000", 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4619e-05 2024-09-21T23:12:44.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.602995+0000", 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1722000000000001e-05 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696210+0000", 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214422000000005 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696354+0000", 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014358799999999999 2024-09-21T23:12:44.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696476+0000", 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012290200000000001 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40167 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3014656~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.27560816800001, 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093735125000000002, 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40167 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603010+0000", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603011+0000", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603016+0000", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603031+0000", 2024-09-21T23:12:44.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5268e-05 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603042+0000", 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1531e-05 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696547+0000", 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093504773999999999 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696650+0000", 2024-09-21T23:12:44.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102592 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696745+0000", 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5342999999999998e-05 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40168 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3072000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.27553410199999, 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094026406000000007, 2024-09-21T23:12:44.889 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40168 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603084+0000", 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603085+0000", 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603097+0000", 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603114+0000", 2024-09-21T23:12:44.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6854000000000002e-05 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603126+0000", 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.252e-05 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696844+0000", 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093717616000000004 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.696986+0000", 2024-09-21T23:12:44.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142251 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697110+0000", 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012402600000000001 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40169 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3129344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.275471376, 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094229777000000001, 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.893 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40169 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603147+0000", 2024-09-21T23:12:44.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603148+0000", 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603153+0000", 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.271e-06 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603168+0000", 2024-09-21T23:12:44.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4823e-05 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603180+0000", 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2302e-05 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697167+0000", 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093986513999999993 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697274+0000", 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010743600000000001 2024-09-21T23:12:44.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697376+0000", 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102328 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40170 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3186688~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.275419426, 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094554206000000002, 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40170 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603199+0000", 2024-09-21T23:12:44.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603347+0000", 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014856900000000001 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603356+0000", 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603376+0000", 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0316999999999999e-05 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603391+0000", 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4994000000000001e-05 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697479+0000", 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094087559000000001 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697623+0000", 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000144695 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697753+0000", 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000129579 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40171 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.27520634, 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094608403999999993, 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40171 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.603412+0000", 2024-09-21T23:12:44.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605167+0000", 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001754916 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605176+0000", 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7850000000000003e-06 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605199+0000", 2024-09-21T23:12:44.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2107999999999999e-05 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605218+0000", 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259e-05 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697820+0000", 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.092601796 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.697927+0000", 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107396 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698020+0000", 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3144000000000001e-05 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40173 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3358720~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.273322315, 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093359914000000002, 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40173 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605296+0000", 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605297+0000", 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1289999999999999e-06 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605302+0000", 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9450000000000001e-06 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605316+0000", 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4525000000000001e-05 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605329+0000", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2277e-05 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698461+0000", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093132751 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698565+0000", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103403 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698656+0000", 2024-09-21T23:12:44.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0884000000000006e-05 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40174 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3416064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.273274265, 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093690610999999993, 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40174 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605344+0000", 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605345+0000", 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-07 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605350+0000", 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605365+0000", 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5441999999999999e-05 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605377+0000", 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2094e-05 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698756+0000", 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093378647999999995 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.698903+0000", 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014713199999999999 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699034+0000", 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013139799999999999 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40175 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.273225958, 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093911588000000004, 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40175 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605392+0000", 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605393+0000", 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605398+0000", 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8319999999999996e-06 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605412+0000", 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.449e-05 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605424+0000", 2024-09-21T23:12:44.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2111e-05 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699102+0000", 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093677222000000004 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699208+0000", 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010654000000000001 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699304+0000", 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5587999999999994e-05 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40176 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3530752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.273178743, 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094240511999999999, 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40176 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605439+0000", 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605440+0000", 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605445+0000", 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605460+0000", 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4256999999999999e-05 2024-09-21T23:12:44.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605472+0000", 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2285e-05 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699408+0000", 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093936243000000003 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699551+0000", 2024-09-21T23:12:44.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143275 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699680+0000", 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012839500000000001 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40177 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3588096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.273114514, 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094446453999999999, 2024-09-21T23:12:44.921 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40177 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605504+0000", 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605505+0000", 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1489999999999999e-06 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605511+0000", 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1619999999999996e-06 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605527+0000", 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605540+0000", 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2639e-05 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699746+0000", 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094206357000000004 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699854+0000", 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010840699999999999 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.699950+0000", 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5656999999999996e-05 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40178 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 3645440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.273062604, 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094737910999999994, 2024-09-21T23:12:44.925 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40178 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.605555+0000", 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609558+0000", 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040026250000000001 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609569+0000", 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0736e-05 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609595+0000", 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6676999999999999e-05 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.609612+0000", 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6595999999999998e-05 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700052+0000", 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090439796000000003 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700179+0000", 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012737800000000001 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.700293+0000", 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114103 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40187 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 499712~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.26846030999999, 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093639897, 2024-09-21T23:12:44.929 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40187 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610158+0000", 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610159+0000", 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610163+0000", 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7709999999999997e-06 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610183+0000", 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9774000000000001e-05 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610197+0000", 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056000000000001e-05 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703397+0000", 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093199926000000002 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703572+0000", 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000174616 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703798+0000", 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022587400000000001 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40188 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 557056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.268405495, 2024-09-21T23:12:44.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093774659999999996, 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40188 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610213+0000", 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610219+0000", 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1590000000000001e-06 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610234+0000", 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5123000000000001e-05 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610245+0000", 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703823+0000", 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.09357799 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703896+0000", 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2749000000000004e-05 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.703987+0000", 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1201999999999999e-05 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.937 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40189 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 614400~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.268357771, 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094199568999999997, 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.938 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40189 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610260+0000", 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610262+0000", 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.311e-06 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610267+0000", 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0969999999999996e-06 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610281+0000", 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4722999999999999e-05 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610294+0000", 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2673999999999999e-05 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704052+0000", 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093757735999999994 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704224+0000", 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172499 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704460+0000", 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000235529 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40190 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 671744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.268290882, 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.094417205000000004, 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40190 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.610327+0000", 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613408+0000", 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003080527 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613418+0000", 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0248000000000001e-05 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613441+0000", 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3028e-05 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613458+0000", 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6762e-05 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704497+0000", 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.091039409000000002 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704622+0000", 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012480600000000001 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.704744+0000", 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012242500000000001 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40198 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1130496~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.264745604, 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093242583000000004, 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40198 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613872+0000", 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613873+0000", 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613878+0000", 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613893+0000", 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4990999999999999e-05 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613904+0000", 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1334e-05 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.706956+0000", 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093051334999999999 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707026+0000", 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0465999999999999e-05 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707115+0000", 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8854e-05 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.950 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40199 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1187840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.26469886300001, 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093501583999999999, 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40199 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613919+0000", 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613920+0000", 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4e-07 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613925+0000", 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-06 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613940+0000", 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613951+0000", 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1551e-05 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707165+0000", 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093214157000000006 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707286+0000", 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121036 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707421+0000", 2024-09-21T23:12:44.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013440899999999999 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:40200 3.17 3:ebcb635c:::rbd_data.1149b90c06ac.0000000000000019:19 [read 1245184~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.264652341, 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.093653217999999996, 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.955 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 40200 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.613966+0000", 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617316+0000", 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033500219999999998 2024-09-21T23:12:44.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617326+0000", 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8539999999999992e-06 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617353+0000", 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7353000000000001e-05 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.617370+0000", 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7237000000000001e-05 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707465+0000", 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.090094760999999995 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707534+0000", 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9511999999999994e-05 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.707619+0000", 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479000000000002e-05 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61537 3.1 3:83c55438:::rbd_object_map.1149904dce8.0000000000000021:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.469956163000006, 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97575118000000005, 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:44.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61537 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408662+0000", 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:44.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408665+0000", 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408680+0000", 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4946999999999999e-05 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408714+0000", 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4725999999999999e-05 2024-09-21T23:12:44.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408739+0000", 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4083e-05 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408842+0000", 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010327099999999999 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:07.408853+0000", 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0798999999999999e-05 2024-09-21T23:12:44.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.383857+0000", 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.97500402600000002 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384059+0000", 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202148 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:08.384413+0000", 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035430699999999999 2024-09-21T23:12:44.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:44.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:44.964 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:44.964 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:44.965 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:44.966 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:44.966 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:44.966 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:44.966 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:44.967 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:44.967 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:44.967 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:44.968 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:44.964+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:45.066 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:45.064+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:45.083 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:45.084 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:45.084 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:45.096 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:45.167 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:45.164+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:45.219 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:45.219 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:45.219 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:45.219 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:45.220 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:45.233 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:45.277 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:45.273+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 102.982545605, 2024-09-21T23:12:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691283431000002, 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:12:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691253681999996, 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:12:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:12:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691232573999997, 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:12:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:12:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691197693000007, 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:12:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:12:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691175283999996, 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:12:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:12:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691153827999997, 2024-09-21T23:12:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691123161999997, 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:12:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691103425999998, 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:12:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691083442000007, 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:12:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691064673, 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691045500000001, 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:12:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:12:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.691026579999999, 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:12:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:12:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:12:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.677917301999997, 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:12:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:12:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.677891435999996, 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.677856813000005, 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:12:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.677830158000006, 2024-09-21T23:12:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.677806713999999, 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.677786506999993, 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:12:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:12:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:12:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 82.677747327000006, 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:12:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.439 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:45.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:45.441 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:12:45.441 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:45.433+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:45.541 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:45.540+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:45.614 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:45.614 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:45.614 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:45.614 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:45.629 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:45.681 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:45.677+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:45.743 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:45.740+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:45.752 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:45.752 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:45.752 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:45.752 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:45.752 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:45.766 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:45.853 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:45.849+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:45.932 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.43809146700001, 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:45.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.438038029, 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:45.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:45.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:45.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.942 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.437987054, 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:45.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.437936123, 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.947 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:45.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.437880812, 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:45.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:45.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:45.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.437811897, 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:45.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:45.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 122.433106403, 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:45.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:45.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.946952892, 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:45.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:45.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:45.970 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.945724971, 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:45.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.945639176, 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:45.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:45.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.944997581, 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:45.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:45.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.944922707, 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:45.984 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:45.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:45.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.944861929, 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:45.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:45.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.944825611, 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:45.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:45.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:45.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 117.14036381699999, 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:45.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.000 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:46.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.000 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:46.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:46.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 106.623825651, 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:46.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.513917569999997, 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:46.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:46.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:46.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.513886001000003, 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.018 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:46.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:46.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.513849997999998, 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:46.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:46.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:46.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:46.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 55.387066691000001, 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:46.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:46.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:46.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:46.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.038 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:46.040 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:46.040 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:46.037+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:46.140 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:46.136+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:46.209 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:46.209 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:46.223 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:46.273 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:46.269+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:46.342 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:46.340+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:46.346 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:46.346 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:46.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:46.442 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:46.440+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:46.538 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:46.538 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:46.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 155.46048741600001, 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:46.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:46.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:46.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:46.544 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.204935921000001, 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:46.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.548 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.204911701, 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.549 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:46.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.204865315000006, 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.553 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.204850092000001, 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193683338, 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193660610999999, 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:46.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193632014000002, 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.569 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:46.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193619810000001, 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:46.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193600818999997, 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193266503000004, 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:46.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:46.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193222376999998, 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.585 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:46.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193198420000002, 2024-09-21T23:12:46.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:46.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:46.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:46.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.193158217000004, 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.597 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.182019026999996, 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.181997073000005, 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:46.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:46.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:46.605 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.181970655000001, 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:46.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:46.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.181944358999999, 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.181921747000004, 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.614 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:46.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:124711 3.18 3:1b770b03:::rbd_object_map.114937d450a8.0000000000000015:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e155)", 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.8921462140000003, 2024-09-21T23:12:46.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80437497499999999, 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124711 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643030+0000", 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3530000000000002e-06 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643041+0000", 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1593000000000001e-05 2024-09-21T23:12:46.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643070+0000", 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8809000000000001e-05 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643087+0000", 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7116e-05 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643155+0000", 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7698000000000001e-05 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643163+0000", 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.446836+0000", 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80367309600000003 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447039+0000", 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020220500000000001 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447402+0000", 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036361199999999997 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:46.623 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:46.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:46.625 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:46.617+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:46.725 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:46.721+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:46.757 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:46.758 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:46.758 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:46.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:46.826 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:46.824+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:46.888 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:46.888 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:46.888 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:46.888 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:46.889 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:46.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:46.927 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:46.921+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:46.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:47.028 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:47.024+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:47.065 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:47.065 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:47.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:47.065 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.239279459000002, 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.066 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.068 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.239208450999996, 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.070 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.239144159999995, 2024-09-21T23:12:47.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:47.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.239113661000005, 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.081 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238965675000003, 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:47.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.086 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238909169999999, 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238854595000006, 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238799377000007, 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:47.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:47.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.098 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238744748000002, 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.099 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:47.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:47.102 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238625597999999, 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:47.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.106 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238548163999994, 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.107 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:47.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:47.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238482542, 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:47.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:47.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238418214000006, 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238357125999997, 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238296661000007, 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.126 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238232463000003, 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.127 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:47.130 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238158455000004, 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.131 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:47.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238121028999998, 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.135 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238074506999993, 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.139 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:47.143 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.238015012000005, 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:47.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:47.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:47.147 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:47.148 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:47.149 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:47.149 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:47.149 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:47.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:47.150 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:47.148+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:47.158 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:47.158 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:47.175 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:47.175 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:47.196 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:47.196 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:47.215 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:47.215 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:47.234 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:47.235 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:47.249 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:47.245+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:47.254 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:47.254 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:47.320 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:47.320 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:47.335 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:47.350 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:47.345+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:47.442 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:12:47.450 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:47.448+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:47.460 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:47.460 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:47.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:47.561 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:47.560+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:47.643 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:47.643 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:47.651 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:47.645+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:47.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:47.752 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:47.745+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:47.824 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:47.825 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:47.838 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:47.853 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:47.849+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:47.954 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:47.952+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:47.962 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:47.962 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:47.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:48.065 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:48.064+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:48.143 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:48.143 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:48.157 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:48.153+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:48.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:48.257 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:48.256+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:48.321 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:48.321 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:48.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:48.358 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:48.353+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:48.459 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:48.456+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:48.505 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:48.506 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:48.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:48.528 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:12:48.560 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:48.560+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:48.661 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:48.657+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:48.687 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:48.687 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:48.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:48.762 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:48.757+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:48.831 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:48.831 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:48.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:48.889 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:48.885+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:48.963 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:48.960+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:49.010 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:49.010 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:49.024 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:49.069 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:49.065+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:49.147 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:49.147 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:49.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:49.165 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:49.164+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:49.266 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:49.261+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:49.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:49.322 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:49.322 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:49.337 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:49.366 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:49.364+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:49.426 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:49.426 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:49.444 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:49.444 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:49.454 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:49.454 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:49.454 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:49.460 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:49.461 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:49.467 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:49.464+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:49.469 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:49.478 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:49.478 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:49.499 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:49.499 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:49.521 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:49.522 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:49.551 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:49.551 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:49.568 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:49.564+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:49.575 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:49.575 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 107.227746381, 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0718478149999999, 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 61281 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367715+0000", 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367719+0000", 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5999999999999998e-06 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367736+0000", 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6756e-05 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367775+0000", 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9483000000000001e-05 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.367803+0000", 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7719e-05 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368829+0000", 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001025963 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:49.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:02.368858+0000", 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9587e-05 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439258+0000", 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.070399197 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439396+0000", 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013825299999999999 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:03.439563+0000", 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000167257 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79808 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 399360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936484207000007, 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062739694999999998, 2024-09-21T23:12:49.602 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79808 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658977+0000", 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658978+0000", 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658986+0000", 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9009999999999994e-06 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658992+0000", 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5379999999999999e-06 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.658999+0000", 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4189999999999999e-06 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721538+0000", 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062538908000000004 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721624+0000", 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6133999999999999e-05 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721717+0000", 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3270999999999995e-05 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79809 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.606 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936454458, 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062906455999999999, 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79809 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659007+0000", 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659008+0000", 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659010+0000", 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659016+0000", 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4670000000000001e-06 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659021+0000", 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9930000000000003e-06 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721749+0000", 2024-09-21T23:12:49.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062727318000000004 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721831+0000", 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2463e-05 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721914+0000", 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2490999999999993e-05 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79810 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936433350000001, 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063078965000000001, 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79810 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659028+0000", 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659029+0000", 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8299999999999998e-07 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659031+0000", 2024-09-21T23:12:49.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0760000000000001e-06 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659037+0000", 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3820000000000001e-06 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659042+0000", 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8160000000000001e-06 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.721943+0000", 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062900662999999996 2024-09-21T23:12:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722021+0000", 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8850000000000006e-05 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722107+0000", 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5795000000000001e-05 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79811 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 433152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936398468999997, 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063183695999999998, 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79811 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659063+0000", 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659064+0000", 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659066+0000", 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5670000000000002e-06 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659073+0000", 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659079+0000", 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722136+0000", 2024-09-21T23:12:49.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063057572000000006 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722185+0000", 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8989000000000001e-05 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722247+0000", 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1774999999999997e-05 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79812 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 444416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936376060000001, 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063411521999999998, 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79812 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659086+0000", 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659088+0000", 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3590000000000002e-06 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659095+0000", 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659100+0000", 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7040000000000002e-06 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722327+0000", 2024-09-21T23:12:49.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063226596999999995 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722417+0000", 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0823999999999996e-05 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722497+0000", 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9758000000000004e-05 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79813 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936354604000002, 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063633827000000004, 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79813 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659107+0000", 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:12:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659117+0000", 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9350000000000006e-06 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659124+0000", 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659130+0000", 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5060000000000003e-06 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722549+0000", 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063419437999999995 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722630+0000", 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0864000000000004e-05 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722741+0000", 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110568 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79814 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 466944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936323938000001, 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063816603, 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79814 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659138+0000", 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659141+0000", 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.385e-06 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659147+0000", 2024-09-21T23:12:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.161e-06 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659151+0000", 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722772+0000", 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063621028999999996 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722856+0000", 2024-09-21T23:12:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3961999999999993e-05 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.722954+0000", 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7880000000000005e-05 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79815 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 478208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936304202000002, 2024-09-21T23:12:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064036827000000004, 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79815 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659157+0000", 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659158+0000", 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659160+0000", 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659166+0000", 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659171+0000", 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.728e-06 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723004+0000", 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063832996000000003 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723096+0000", 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1267000000000006e-05 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723194+0000", 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8651000000000001e-05 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79816 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 489472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936284217999997, 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064259652, 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79816 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659177+0000", 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659178+0000", 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-21T23:12:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659180+0000", 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9759999999999998e-06 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659185+0000", 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659190+0000", 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5369999999999999e-06 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723239+0000", 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064049141000000004 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723334+0000", 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5575000000000006e-05 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723437+0000", 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102692 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79817 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 500736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936265449000004, 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064488214000000002, 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79817 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659196+0000", 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659197+0000", 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659199+0000", 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1890000000000001e-06 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659205+0000", 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.789e-06 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659209+0000", 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7060000000000003e-06 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723485+0000", 2024-09-21T23:12:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064275341 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723580+0000", 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5189e-05 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723684+0000", 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104626 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79818 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 512000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936246276000006, 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064742644000000002, 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79818 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659215+0000", 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659216+0000", 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.89e-07 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659218+0000", 2024-09-21T23:12:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659224+0000", 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9039999999999997e-06 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659228+0000", 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5140000000000003e-06 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723737+0000", 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064508933000000004 2024-09-21T23:12:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723870+0000", 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013316599999999999 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.723958+0000", 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7744999999999994e-05 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:79819 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 523264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.936227356000003, 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064981408000000004, 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 79819 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659234+0000", 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659235+0000", 2024-09-21T23:12:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659237+0000", 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0030000000000001e-06 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659242+0000", 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8510000000000004e-06 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.659254+0000", 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1771e-05 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724009+0000", 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064755123999999997 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724101+0000", 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2003000000000007e-05 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.724216+0000", 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011430900000000001 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80069 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 445440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.923118078000002, 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062898507000000006, 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80069 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672344+0000", 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4799999999999998e-07 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672347+0000", 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9799999999999998e-06 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672355+0000", 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0320000000000003e-06 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672361+0000", 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9399999999999999e-06 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735039+0000", 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062677989000000003 2024-09-21T23:12:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735120+0000", 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0406000000000002e-05 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735242+0000", 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122512 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80070 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 456704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.923092212, 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063118292000000006, 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80070 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672369+0000", 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672370+0000", 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672373+0000", 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672380+0000", 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5580000000000003e-06 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672395+0000", 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4849e-05 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735290+0000", 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062894607000000005 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735372+0000", 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2291000000000002e-05 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735488+0000", 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115844 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80071 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 467968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.923057588999995, 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063313214000000007, 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80071 2024-09-21T23:12:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672404+0000", 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672405+0000", 2024-09-21T23:12:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5799999999999999e-07 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672407+0000", 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672417+0000", 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1679999999999993e-06 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672423+0000", 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.156e-06 2024-09-21T23:12:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735518+0000", 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063095555999999997 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735600+0000", 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1552999999999997e-05 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735717+0000", 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117404 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80072 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 479232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.923030933999996, 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063513636999999998, 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80072 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672431+0000", 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.32e-07 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672434+0000", 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.593e-06 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672441+0000", 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2219999999999996e-06 2024-09-21T23:12:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672446+0000", 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735757+0000", 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063311086000000003 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735837+0000", 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0018999999999994e-05 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735944+0000", 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106949 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80073 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 490496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.923007490000003, 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063675420999999996, 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80073 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672454+0000", 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672456+0000", 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.756e-06 2024-09-21T23:12:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672461+0000", 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3039999999999998e-06 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672467+0000", 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2500000000000003e-06 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.735978+0000", 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063511208 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736058+0000", 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0142999999999995e-05 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736130+0000", 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1432000000000003e-05 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80074 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.922987282999998, 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063818343, 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80074 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672474+0000", 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672475+0000", 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672477+0000", 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3309999999999998e-06 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672484+0000", 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1550000000000001e-06 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672490+0000", 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736157+0000", 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063667709000000003 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736232+0000", 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4389999999999998e-05 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736293+0000", 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0971000000000003e-05 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:80075 3.0 3:04934851:::rbd_data.11498e015c32.0000000000000000:27 [read 513024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.922948102999996, 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063931164999999998, 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 80075 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672514+0000", 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5700000000000002e-07 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672517+0000", 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0019999999999998e-06 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672525+0000", 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7789999999999996e-06 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.672530+0000", 2024-09-21T23:12:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736321+0000", 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063791009999999995 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736377+0000", 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5216999999999997e-05 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.736445+0000", 2024-09-21T23:12:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8016999999999996e-05 2024-09-21T23:12:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:49.683 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:49.683 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:49.683 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:49.685 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T23:12:49.685 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-21T23:12:49.687 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:49.687 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:49.681+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:49.788 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:49.781+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:49.856 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:49.856 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:49.856 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:49.856 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:49.873 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:49.896 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:49.897 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:49.897 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:49.889+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:49.899 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:49.899 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:49.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-21T23:12:49.989 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:49.985+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:50.010 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:50.010 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:50.010 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:50.025 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:50.088 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:50.089 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:50.090 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:50.088+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:50.094 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:50.094 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:50.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.691837975, 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:50.185 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:50.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:50.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.691784537, 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.190 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:50.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:50.194 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.691733562, 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:50.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:50.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.69168263100001, 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.69162731999999, 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.69155840499999, 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.686852911, 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.2006994, 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.199471479, 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.19938568400001, 2024-09-21T23:12:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.198744089, 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.198669215, 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.198608437, 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 126.198572119, 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 121.394110325, 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 110.877572159, 2024-09-21T23:12:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.767664078000003, 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.767632509000002, 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.767596505999997, 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.640813199, 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.276 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:50.277 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:50.278 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:50.276+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:50.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:50.298 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:50.299 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-21T23:12:50.389 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:50.385+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:50.392 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:50.392 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:50.392 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:50.392 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:50.411 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:50.497 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:50.493+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:50.505 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:50.508 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:50.508 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:50.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-21T23:12:50.565 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:12:50.583 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:50.584 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:50.584 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:50.577+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:50.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:50.680 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:50.676+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:50.710 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:50.711 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:50.713 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:50.713 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:50.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-21T23:12:50.764 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:50.764 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 159.68895239, 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:50.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:50.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:50.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.433400895000005, 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:50.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.433376675000005, 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:50.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.433330288999997, 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.778 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:50.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:50.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.781 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.433315066000006, 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.782 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:50.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.422148312000004, 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:50.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.422125585000003, 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:50.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:50.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:50.793 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.422096988000007, 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.422084784000006, 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:50.797 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:50.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:50.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.422065793000002, 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.801 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:50.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.421731476999994, 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:50.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:50.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.421687351000003, 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:50.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.421663394000007, 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:50.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.421623190999995, 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.410484001, 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.410462046999996, 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:50.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:50.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:50.828 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.410435629000006, 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:50.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.410409333000004, 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:50.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:50.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:50.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.410386720999995, 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.837 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:124711 3.18 3:1b770b03:::rbd_object_map.114937d450a8.0000000000000015:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e155)", 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.120611188, 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80437497499999999, 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124711 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643030+0000", 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3530000000000002e-06 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643041+0000", 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1593000000000001e-05 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643070+0000", 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8809000000000001e-05 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:50.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643087+0000", 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7116e-05 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643155+0000", 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7698000000000001e-05 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643163+0000", 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.446836+0000", 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80367309600000003 2024-09-21T23:12:50.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447039+0000", 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020220500000000001 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447402+0000", 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036361199999999997 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:50.846 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:50.846 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:50.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:50.847 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:50.844+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:50.861 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:50.861 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:50.861 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:50.861 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:50.862 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:50.864 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:50.864 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:50.865 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-21T23:12:50.947 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:50.944+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:50.979 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:50.979 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:50.980 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:50.980 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:50.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:51.048 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:51.041+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:51.055 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:51.056 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:51.056 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:51.056 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:51.058 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:51.058 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:51.058 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-21T23:12:51.149 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:51.148+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:51.159 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:51.159 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:51.159 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:51.159 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:51.159 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:51.173 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:no valid command found; 10 closest matches: 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:0 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:1 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:2 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:abort 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:assert 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:bench [] [] [] [] 2024-09-21T23:12:51.245 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:12:51.246 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs files list 2024-09-21T23:12:51.246 INFO:teuthology.orchestra.run.smithi150.stderr:bluefs stats 2024-09-21T23:12:51.246 INFO:teuthology.orchestra.run.smithi150.stderr:bluestore allocator dump block 2024-09-21T23:12:51.246 INFO:teuthology.orchestra.run.smithi150.stderr:Error EINVAL: invalid command 2024-09-21T23:12:51.249 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:12:51.249 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:12:51.250 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:51.245+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:51.344 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:51.344 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:51.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:51.344 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:51.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517971873999997, 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:51.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.348 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517900866000005, 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517836575000004, 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:51.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.356 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517806075999999, 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:51.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:51.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517658089999998, 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517601584999994, 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.365 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:51.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:51.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:51.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517547010000001, 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:51.369 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517491792000001, 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.373 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:51.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:51.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517437162999997, 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.377 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517318012999993, 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.381 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517240579000003, 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.385 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:51.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:51.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517174956999995, 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.389 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:51.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.392 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517110629000001, 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.517049541000006, 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:51.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:51.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.400 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.516989076000002, 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:51.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.516924877999998, 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:51.404 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:51.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.516850869999999, 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.408 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.516813443999993, 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:51.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.516766922000002, 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:51.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.516707427, 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:51.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:51.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:51.423 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:51.424 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:51.425 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:51.421+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:51.525 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:51.521+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:51.534 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:12:51.566 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:51.566 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:51.566 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:51.566 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:51.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:51.595 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:51.629 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:51.628+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:51.719 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:51.734 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:51.734 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:51.734 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:51.734 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:51.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:51.738 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:51.732+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:51.761 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:51.761 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:51.778 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:51.778 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:51.793 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:51.793 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:51.810 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:51.810 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:51.827 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:51.827 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:51.836 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:51.832+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:51.845 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:51.845 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:51.864 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:51.864 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:51.883 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:51.883 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:51.905 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:51.905 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:51.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:51.961 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:51.960+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:52.037 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:52.036+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:52.085 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:52.085 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:52.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:52.141 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:52.140+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:52.240 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:52.233+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:52.264 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:52.264 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:52.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:52.341 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:52.337+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:52.403 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:52.403 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:52.417 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:52.465 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:52.464+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:52.543 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:52.543 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:52.543 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:52.540+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:52.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:52.644 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:52.637+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:52.745 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:52.741+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:52.746 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:52.746 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:52.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:52.845 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:52.844+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:52.889 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:52.889 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:52.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:52.949 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:52.945+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:53.035 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:53.035 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:53.046 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:53.041+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:53.049 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:53.147 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:53.144+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:53.217 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:53.217 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:53.231 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:53.247 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:53.244+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:53.348 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:53.341+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:53.354 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:53.354 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:53.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:53.448 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:53.448+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:53.537 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:53.537 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:53.549 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:53.548+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:53.552 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:53.569 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:12:53.650 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:53.645+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:53.680 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:53.680 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:53.697 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:53.751 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:53.748+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:53.852 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:53.845+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:53.859 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:53.859 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:53.874 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:53.900 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:53.952 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:53.948+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:54.035 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:54.035 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:54.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:54.052 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:54.045+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:54.056 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:54.056 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:54.073 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:54.073 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:54.089 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:54.090 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:54.106 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:54.106 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:54.126 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:54.127 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:54.147 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:54.147 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:54.154 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:54.149+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:54.167 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:54.167 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:54.167 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:54.167 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:54.168 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:54.181 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:54.188 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:54.189 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:54.254 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:54.249+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39785 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8462336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:54.307 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.81265033299999, 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019789115999999999, 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39785 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:54.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491547+0000", 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491548+0000", 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.06e-06 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491553+0000", 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3630000000000002e-06 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491570+0000", 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6892999999999999e-05 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491583+0000", 2024-09-21T23:12:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2616000000000001e-05 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510708+0000", 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019125000999999999 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.510898+0000", 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019049599999999999 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511336+0000", 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043768700000000003 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39804 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8404992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.81259689500001, 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020432236999999999, 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39804 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491600+0000", 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491601+0000", 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491606+0000", 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491623+0000", 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6745999999999999e-05 2024-09-21T23:12:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491635+0000", 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2469999999999999e-05 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511404+0000", 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019768601 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.511617+0000", 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021349499999999999 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512032+0000", 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041483200000000002 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39808 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8077312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.81254591999999, 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021209311000000002, 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39808 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491651+0000", 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491652+0000", 2024-09-21T23:12:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491657+0000", 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491674+0000", 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6714999999999998e-05 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491686+0000", 2024-09-21T23:12:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.255e-05 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512154+0000", 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020468218 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512430+0000", 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027602699999999998 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512860+0000", 2024-09-21T23:12:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042984099999999998 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39809 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8019968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.81249498899999, 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021914499, 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39809 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491702+0000", 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491703+0000", 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0729999999999999e-06 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491708+0000", 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491725+0000", 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6639000000000001e-05 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491740+0000", 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5849999999999999e-05 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.512956+0000", 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021215523999999999 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513212+0000", 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025622699999999998 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513616+0000", 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00040426900000000002 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39821 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8167424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.812439678, 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022285748000000001, 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39821 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491757+0000", 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491758+0000", 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491778+0000", 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9491999999999999e-05 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491796+0000", 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8162999999999999e-05 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491809+0000", 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3083e-05 2024-09-21T23:12:54.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513713+0000", 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021903456000000002 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.513840+0000", 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000127119 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514043+0000", 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020328100000000001 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39822 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.81237076299999, 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.023018166999999999, 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39822 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491826+0000", 2024-09-21T23:12:54.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491827+0000", 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.04e-06 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491832+0000", 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0130000000000001e-06 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491849+0000", 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.491862+0000", 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2583e-05 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514141+0000", 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022279021999999999 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514375+0000", 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023373799999999999 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.514844+0000", 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000469712 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:39836 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 8372224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:54.334 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.80766526900001, 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019858240999999999, 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 39836 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:54.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496532+0000", 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496533+0000", 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496539+0000", 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9200000000000001e-06 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496558+0000", 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9721999999999999e-05 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.496573+0000", 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4663e-05 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516068+0000", 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019495235 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516198+0000", 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012922899999999999 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.516390+0000", 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019227900000000001 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.338 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41255 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9224192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.32151175800001, 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.022452857999999999, 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41255 2024-09-21T23:12:54.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982685+0000", 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982687+0000", 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982705+0000", 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7459999999999999e-05 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982733+0000", 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.847e-05 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.982752+0000", 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9402000000000002e-05 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997163+0000", 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014410414999999999 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.997270+0000", 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107496 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005138+0000", 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078676130000000007 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41346 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9469952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.32028383700001, 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019875983, 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41346 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983913+0000", 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983914+0000", 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.082e-06 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983920+0000", 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5480000000000001e-06 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983937+0000", 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6833e-05 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983948+0000", 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.098e-05 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003476+0000", 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019528496999999999 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003594+0000", 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011777000000000001 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003789+0000", 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019527300000000001 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41347 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9527296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:54.347 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.32019804199999, 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020037454999999999, 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41347 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:54.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.983999+0000", 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984000+0000", 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.54e-07 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984005+0000", 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984019+0000", 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3892e-05 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984028+0000", 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4800000000000007e-06 2024-09-21T23:12:54.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003833+0000", 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019804755 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.003910+0000", 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7087000000000002e-05 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004036+0000", 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126346 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.351 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41361 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9674752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.319556447, 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019896496999999999, 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41361 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.352 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984640+0000", 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984642+0000", 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.285e-06 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984649+0000", 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984668+0000", 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9936e-05 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984692+0000", 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3065e-05 2024-09-21T23:12:54.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004286+0000", 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019594519000000001 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004390+0000", 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010435200000000001 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004537+0000", 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014655700000000001 2024-09-21T23:12:54.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41373 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9707520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.31948157299999, 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020145435, 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41373 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984715+0000", 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984717+0000", 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984723+0000", 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1249999999999998e-06 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984742+0000", 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8890999999999999e-05 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984756+0000", 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3930999999999999e-05 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004578+0000", 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019821958000000001 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004661+0000", 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3411999999999994e-05 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004861+0000", 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019929799999999999 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41374 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9764864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.31942079500001, 2024-09-21T23:12:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020474875999999999, 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41374 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984776+0000", 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984777+0000", 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2309999999999999e-06 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984783+0000", 2024-09-21T23:12:54.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984795+0000", 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2131e-05 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984802+0000", 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.289e-06 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.004922+0000", 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020119926999999999 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005030+0000", 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107594 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005251+0000", 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000221373 2024-09-21T23:12:54.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:41375 3.9 3:92469328:::rbd_data.1149ed0ffb75.0000000000000003:18 [read 9822208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 130.319384477, 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020907445, 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 41375 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984812+0000", 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984813+0000", 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1200000000000003e-07 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984816+0000", 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.134e-06 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984825+0000", 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:43.984831+0000", 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6370000000000002e-06 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005320+0000", 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020488909 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005481+0000", 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016079700000000001 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:44.005720+0000", 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000238848 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:53892 3.18 3.a2ec9878 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:54.369 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 125.51492268299999, 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0040374270000001, 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 53892 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789274+0000", 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789276+0000", 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.748e-06 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789285+0000", 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.081e-06 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789303+0000", 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7944e-05 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.789314+0000", 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0934999999999999e-05 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790059+0000", 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00074540399999999997 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:48.790068+0000", 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793295+0000", 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0032269199999999 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:49.793312+0000", 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6495e-05 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:60849 3.9 3:961e1ced:::rbd_data.114952cb5a61.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 114.99838451700001, 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043733266, 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 60849 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305812+0000", 2024-09-21T23:12:54.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305815+0000", 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7300000000000001e-06 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305827+0000", 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.224e-05 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305859+0000", 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1556000000000001e-05 2024-09-21T23:12:54.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305873+0000", 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.305997+0000", 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012369700000000001 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.306136+0000", 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013893999999999999 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349393+0000", 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043257421999999997 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349451+0000", 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7534999999999997e-05 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:59.349546+0000", 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4827000000000005e-05 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.378 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116335 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 63.888476435999998, 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.020983927999999999, 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116335 2024-09-21T23:12:54.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415720+0000", 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415721+0000", 2024-09-21T23:12:54.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415725+0000", 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7529999999999999e-06 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415736+0000", 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415743+0000", 2024-09-21T23:12:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428701+0000", 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012958485 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.428954+0000", 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025238799999999998 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.430000+0000", 2024-09-21T23:12:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00104649 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436216+0000", 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062155810000000004 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436406+0000", 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000190395 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436584+0000", 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017833899999999999 2024-09-21T23:12:54.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436704+0000", 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119997 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116336 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 63.888444866999997, 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.021219104999999999, 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.384 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116336 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415752+0000", 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415753+0000", 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9699999999999995e-07 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415757+0000", 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415766+0000", 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2390000000000008e-06 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415773+0000", 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8240000000000002e-06 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432004+0000", 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016231755 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432206+0000", 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000201685 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.432860+0000", 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00065428600000000002 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436364+0000", 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035039950000000002 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436730+0000", 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036596699999999997 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436864+0000", 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000134084 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.436971+0000", 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010668 2024-09-21T23:12:54.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116337 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 63.888408863999999, 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024766699999999999, 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116337 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415788+0000", 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415789+0000", 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415793+0000", 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6959999999999996e-06 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415803+0000", 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5729999999999997e-06 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.415810+0000", 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434207+0000", 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018397430999999999 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.434424+0000", 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021694499999999999 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.435094+0000", 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066929999999999995 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.439601+0000", 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045070919999999999 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440488+0000", 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00088719999999999999 2024-09-21T23:12:54.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440528+0000", 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0269999999999999e-05 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.440555+0000", 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6512999999999998e-05 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:116405 3.18 3:196e0317:::rbd_object_map.114969d6b98:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-21T23:12:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 63.761625557000002, 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.20622121400000001, 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 116405 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542571+0000", 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542573+0000", 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.905e-06 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542581+0000", 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000004e-06 2024-09-21T23:12:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542599+0000", 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7756000000000001e-05 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542608+0000", 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.533e-06 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542662+0000", 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4162000000000001e-05 2024-09-21T23:12:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542733+0000", 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0259000000000006e-05 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.542978+0000", 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024528899999999999 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.544214+0000", 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012364159999999999 2024-09-21T23:12:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748628+0000", 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.204413658 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748723+0000", 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4687999999999999e-05 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:50.748793+0000", 2024-09-21T23:12:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9978e-05 2024-09-21T23:12:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.401 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:54.402 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:54.403 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:54.397+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:54.503 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:54.497+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:54.571 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is set 2024-09-21T23:12:54.580 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:54.580 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:54.598 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:54.641 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:54.637+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:54.704 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:54.697+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:54.766 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:54.766 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:54.766 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:54.766 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:54.766 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:54.781 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:54.805 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:54.804+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:54.907 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:54.904+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 163.87323511, 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:54.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:54.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.617683615000004, 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.617659395000004, 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:54.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:54.965 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.617613008999996, 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:54.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.969 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.617597786000005, 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.970 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:54.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.606431032000003, 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:54.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.606408305000002, 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.982 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.606379708000006, 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:54.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:54.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.986 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.606367504000005, 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.987 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:54.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:54.990 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.606348513, 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.991 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:54.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:54.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.606014196999993, 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.605970071000002, 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.000 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.605946114000005, 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.605905910999994, 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.594766720999999, 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.594744766999995, 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.594718349000004, 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.594692053000003, 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.594669440999994, 2024-09-21T23:12:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:124711 3.18 3:1b770b03:::rbd_object_map.114937d450a8.0000000000000015:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e155)", 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.304893908, 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80437497499999999, 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124711 2024-09-21T23:12:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643030+0000", 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3530000000000002e-06 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643041+0000", 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1593000000000001e-05 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643070+0000", 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8809000000000001e-05 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643087+0000", 2024-09-21T23:12:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7116e-05 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643155+0000", 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7698000000000001e-05 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643163+0000", 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.446836+0000", 2024-09-21T23:12:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80367309600000003 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447039+0000", 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020220500000000001 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447402+0000", 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036361199999999997 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:55.039 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:55.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:55.040 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:55.037+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:55.141 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:55.141+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:55.171 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:55.171 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:55.171 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:55.171 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:55.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:55.241 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:55.240+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:55.303 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:55.303 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:55.304 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:55.304 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:55.304 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:55.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:55.342 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:55.337+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:55.443 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:55.440+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:55.491 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:55.491 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.664243606, 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.492 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.66417259799999, 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.496 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:55.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.500 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.66410830700001, 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.502 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.504 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.664077808, 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:55.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.508 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663929822, 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.509 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:55.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:55.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663873317, 2024-09-21T23:12:55.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.514 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.515 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.516 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:55.517 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663818742, 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.518 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.520 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.521 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663763524, 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:55.522 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.523 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.524 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:55.525 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663708895, 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.526 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.527 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:55.528 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:55.529 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663589745, 2024-09-21T23:12:55.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.534 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.66351231100001, 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:55.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:55.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.538 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663446689, 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.539 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:55.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:55.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:55.542 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663382361, 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.543 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.66332127299999, 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.547 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:55.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.550 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:55.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:55.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.550 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.552 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663260808, 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:55.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:55.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.556 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.66319661, 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:55.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663122602, 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:55.561 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:55.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:55.565 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663085176, 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.566 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:55.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:55.569 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.663038654, 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:55.570 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.572 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:55.574 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 102.662979159, 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:55.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:55.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.578 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:55.579 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:55.579 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:55.580 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:55.581 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:55.576+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:55.681 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:55.681+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:55.756 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:55.756 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:55.756 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:55.756 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:55.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:55.817 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:55.816+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:55.883 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:55.881+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:55.893 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:55.893 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:55.893 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:55.893 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:55.894 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:55.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:12:55.993 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:55.992+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:56.070 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:56.070 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:56.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:56.070 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:56.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.070 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 137.33933668200001, 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:12:56.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:12:56.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:12:56.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:12:56.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.076 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.692023188999997, 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691990513999997, 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:12:56.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:12:56.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691967179000002, 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.085 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:12:56.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691934614000004, 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:12:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:12:56.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.093 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691846991999995, 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.095 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691728244999993, 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:12:56.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:56.102 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691703480000001, 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:56.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.106 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691682333000003, 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691624636, 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:12:56.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:12:56.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691589754999995, 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.116 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:12:56.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691564229999997, 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.120 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:12:56.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691493043999998, 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:12:56.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691457197999995, 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:12:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691440104999998, 2024-09-21T23:12:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:12:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:12:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691424093999998, 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691366845999994, 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:12:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:12:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691285480999994, 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:12:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:12:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691253931000006, 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:12:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:12:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:12:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:12:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 93.691107033999998, 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:12:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:12:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:12:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:56.159 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:56.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:12:56.161 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:56.157+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:56.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:12:56.250 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T23:12:56.250 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:12:56.250 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T23:12:56.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-21T23:12:56.301 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:56.297+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:56.330 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:56.331 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:56.331 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:56.331 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:56.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:12:56.363 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:56.363 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:56.380 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:56.380 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:56.392 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:56.392+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:56.401 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:56.401 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:56.418 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:56.418 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:56.437 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:56.438 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:56.457 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:56.457 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:56.463 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:56.457+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:56.464 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:56.464 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:56.464 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:56.464 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:56.464 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:56.477 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:56.477 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:56.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:12:56.495 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:56.495 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:56.565 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:56.564+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:56.567 INFO:tasks.ceph.mon.a.smithi150.stderr:2024-09-21T23:12:56.561+0000 7f4d03887640 -1 mon.a@0(leader).osd e174 definitely_dead 0 2024-09-21T23:12:56.580 INFO:teuthology.orchestra.run.smithi150.stderr:marked out osd.6. 2024-09-21T23:12:56.595 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:12:56.658 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:56.658 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:56.665 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:56.664+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:56.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:12:56.766 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:56.764+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:56.838 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:56.839 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:56.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:12:56.867 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:56.861+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:56.969 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:56.969+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:57.019 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:57.019 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:57.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:12:57.081 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:57.080+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:57.158 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:57.158 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:57.170 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:57.169+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:57.172 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:12:57.271 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:57.265+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:57.305 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:57.306 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:57.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:12:57.372 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:57.368+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:57.473 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:57.469+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:57.479 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:57.479 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:57.493 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:12:57.574 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:57.572+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:57.590 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is set 2024-09-21T23:12:57.664 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:57.664 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:57.675 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:57.672+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:57.678 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:12:57.776 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:57.769+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:57.799 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:57.799 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:57.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:12:57.878 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:57.876+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:57.980 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:57.977+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:57.980 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:57.980 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:57.994 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:12:58.081 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T23:12:58.080+0000 7fddd9638640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 35740) UID: 0 2024-09-21T23:12:58.162 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:58.162 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:58.176 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:12:58.221 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:58.221+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:58.282 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:58.280+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:58.298 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:58.299 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:58.313 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:12:58.383 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:58.380+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:58.481 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:58.481 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:58.484 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:58.480+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:58.495 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:12:58.512 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:12:58.584 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T23:12:58.584+0000 7f2d709ad640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26599) UID: 0 2024-09-21T23:12:58.628 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-21T23:12:58.628 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:12:58.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:12:58.688 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:58.689+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:58.694 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0: {} 2024-09-21T23:12:58.694 INFO:teuthology.orchestra.run.smithi150.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:12:58.711 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1: {} 2024-09-21T23:12:58.712 INFO:teuthology.orchestra.run.smithi150.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:12:58.730 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2: {} 2024-09-21T23:12:58.730 INFO:teuthology.orchestra.run.smithi150.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:12:58.748 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3: {} 2024-09-21T23:12:58.748 INFO:teuthology.orchestra.run.smithi150.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:12:58.769 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4: {} 2024-09-21T23:12:58.770 INFO:teuthology.orchestra.run.smithi150.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:12:58.774 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:58.774 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-21T23:12:58.774 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:58.786 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:58.785+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:58.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:12:58.791 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5: {} 2024-09-21T23:12:58.791 INFO:teuthology.orchestra.run.smithi150.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:12:58.813 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6: {} 2024-09-21T23:12:58.814 INFO:teuthology.orchestra.run.smithi150.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T23:12:58.835 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7: {} 2024-09-21T23:12:58.836 INFO:teuthology.orchestra.run.smithi150.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T23:12:58.887 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T23:12:58.884+0000 7f06c66c1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26598) UID: 0 2024-09-21T23:12:58.908 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-21T23:12:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-21T23:12:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-21T23:12:58.908 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:58.923 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:12:58.988 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:58.985+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:11785 3.18 3:1a49ebd8:::rbd_data.1149e13ad964.0000000000000052:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 167.98951071600001, 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063322209000000004, 2024-09-21T23:12:59.066 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 11785 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074686+0000", 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074688+0000", 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.796e-06 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074696+0000", 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:12:59.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074715+0000", 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9014000000000001e-05 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074725+0000", 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0162e-05 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.074822+0000", 2024-09-21T23:12:59.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6878000000000003e-05 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.075028+0000", 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000206558 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 0", 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.084471+0000", 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0094428470000000007 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:12:59.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137745+0000", 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053273762000000002 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137791+0000", 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6595999999999999e-05 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.137880+0000", 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8459000000000001e-05 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:10:11.138008+0000", 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012837300000000001 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76765 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3635200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.733959221000006, 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059495434999999999, 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76765 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330237+0000", 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330238+0000", 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330240+0000", 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330246+0000", 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3219999999999998e-06 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330253+0000", 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1439999999999997e-06 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389555+0000", 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059301390000000002 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389633+0000", 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8114000000000005e-05 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389733+0000", 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010005099999999999 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.076 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76766 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3646464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.733935001000006, 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059687925000000003, 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76766 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330262+0000", 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330265+0000", 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6510000000000001e-06 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330280+0000", 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5633000000000001e-05 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330286+0000", 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389766+0000", 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059480334000000003 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389849+0000", 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3248e-05 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389950+0000", 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010036599999999999 2024-09-21T23:12:59.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76767 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3657728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.733888614999998, 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059856398999999998, 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76767 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330308+0000", 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330310+0000", 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330316+0000", 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2379999999999997e-06 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330319+0000", 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.168e-06 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.389982+0000", 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059663712000000001 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390067+0000", 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4314999999999995e-05 2024-09-21T23:12:59.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390164+0000", 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7725000000000004e-05 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:76768 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3668992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.733873392000007, 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060215844999999997, 2024-09-21T23:12:59.085 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 76768 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330323+0000", 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330324+0000", 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.28e-07 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330325+0000", 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.432e-06 2024-09-21T23:12:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330328+0000", 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.330334+0000", 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0109999999999997e-06 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390352+0000", 2024-09-21T23:12:59.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060018086999999998 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390439+0000", 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6249999999999996e-05 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.390539+0000", 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010052000000000001 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77016 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722706638000005, 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059521814999999999, 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77016 2024-09-21T23:12:59.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341490+0000", 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341491+0000", 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.58e-07 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341493+0000", 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7879999999999998e-06 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341502+0000", 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3049999999999993e-06 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341507+0000", 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0150000000000003e-06 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400818+0000", 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05931152 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.400896+0000", 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7576000000000004e-05 2024-09-21T23:12:59.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401012+0000", 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116153 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77017 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722683911000004, 2024-09-21T23:12:59.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059843355000000001, 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77017 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341513+0000", 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341524+0000", 2024-09-21T23:12:59.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1283e-05 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341532+0000", 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341535+0000", 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3469999999999999e-06 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401070+0000", 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05953518 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401154+0000", 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3848000000000001e-05 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401356+0000", 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020219899999999999 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77018 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722655313999994, 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060003177999999997, 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77018 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.099 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341541+0000", 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341542+0000", 2024-09-21T23:12:59.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341543+0000", 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5519999999999999e-06 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341547+0000", 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5860000000000001e-06 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341550+0000", 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.712e-06 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401394+0000", 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059844096999999999 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401451+0000", 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7114999999999998e-05 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401545+0000", 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77019 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722643110000007, 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060185081000000001, 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.103 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.104 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77019 2024-09-21T23:12:59.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.105 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341554+0000", 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341556+0000", 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.792e-06 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341561+0000", 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341566+0000", 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4959999999999998e-06 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401605+0000", 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060039593000000002 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401660+0000", 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4934e-05 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401739+0000", 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8257999999999995e-05 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.108 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77020 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722624119000002, 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060323545999999999, 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77020 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341573+0000", 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2899999999999999e-07 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341575+0000", 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3819999999999998e-06 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341583+0000", 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341594+0000", 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.131e-05 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401772+0000", 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060177277000000001 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401820+0000", 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8038000000000003e-05 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401896+0000", 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6581000000000004e-05 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77021 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722289802999995, 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060115671000000002, 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77021 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341907+0000", 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341908+0000", 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341911+0000", 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7780000000000001e-06 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341925+0000", 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3740000000000001e-05 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341932+0000", 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.922e-06 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401925+0000", 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059993066999999997 2024-09-21T23:12:59.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.401969+0000", 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3829000000000003e-05 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402023+0000", 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3554999999999998e-05 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77022 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722245677000004, 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060218800000000003, 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77022 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341951+0000", 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341952+0000", 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341954+0000", 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341962+0000", 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4950000000000002e-06 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341967+0000", 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4870000000000002e-06 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402050+0000", 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060082271999999999 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402092+0000", 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2231999999999998e-05 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402170+0000", 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8029999999999997e-05 2024-09-21T23:12:59.121 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77023 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722221719999993, 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060339639, 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77023 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341975+0000", 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341977+0000", 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1169999999999998e-06 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.341999+0000", 2024-09-21T23:12:59.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1087999999999999e-05 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342006+0000", 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6329999999999996e-06 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402207+0000", 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060200397000000003 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402256+0000", 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9441999999999999e-05 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402315+0000", 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8585999999999998e-05 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77024 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.126 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.722181516999996, 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060460200999999998, 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77024 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:59.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342015+0000", 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342016+0000", 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342019+0000", 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9809999999999999e-06 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342024+0000", 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.342030+0000", 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402344+0000", 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060314785000000003 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402415+0000", 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0199999999999999e-05 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.402475+0000", 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0856000000000001e-05 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.130 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77277 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3614720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.711042327000001, 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059481846999999997, 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77277 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353154+0000", 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353155+0000", 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353158+0000", 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7420000000000002e-06 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353164+0000", 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-21T23:12:59.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353170+0000", 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412458+0000", 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059288718999999997 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412535+0000", 2024-09-21T23:12:59.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6378999999999995e-05 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412636+0000", 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010149600000000001 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77278 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3625984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.711020372999997, 2024-09-21T23:12:59.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059680245, 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77278 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:59.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353176+0000", 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353177+0000", 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353184+0000", 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353191+0000", 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6320000000000002e-06 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353196+0000", 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412668+0000", 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059471362 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412755+0000", 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7692e-05 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412857+0000", 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101265 2024-09-21T23:12:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77279 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3637248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.710993955000006, 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059868809000000002, 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77279 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353203+0000", 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353209+0000", 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8189999999999997e-06 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353216+0000", 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1899999999999998e-06 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353221+0000", 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1050000000000003e-06 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412891+0000", 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059669792999999999 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.412976+0000", 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4599999999999996e-05 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413072+0000", 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5902000000000004e-05 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77280 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3648512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.144 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.710967659000005, 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060066459000000003, 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77280 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353229+0000", 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9999999999999998e-07 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353232+0000", 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.373e-06 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353238+0000", 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353243+0000", 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0409999999999997e-06 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413111+0000", 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.059867464000000002 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413199+0000", 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8141999999999997e-05 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413296+0000", 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6422000000000006e-05 2024-09-21T23:12:59.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:77281 3.2 3:47e27851:::rbd_data.11498e015c32.0000000000000002:27 [read 3659776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.710945046999996, 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060254051000000003, 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 77281 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353252+0000", 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6199999999999998e-07 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353254+0000", 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.356e-06 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353261+0000", 2024-09-21T23:12:59.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.37e-06 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.353266+0000", 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413328+0000", 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.060061852999999998 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413412+0000", 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:11:22.413506+0000", 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3358999999999998e-05 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4425.0:124711 3.18 3:1b770b03:::rbd_object_map.114937d450a8.0000000000000015:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e155)", 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.421169513999999, 2024-09-21T23:12:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80437497499999999, 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4425", 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124711 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643027+0000", 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643030+0000", 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3530000000000002e-06 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-21T23:12:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643041+0000", 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1593000000000001e-05 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643070+0000", 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8809000000000001e-05 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643087+0000", 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7116e-05 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643155+0000", 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7698000000000001e-05 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:36.643163+0000", 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4929999999999998e-06 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.446836+0000", 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.80367309600000003 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447039+0000", 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020220500000000001 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-21T23:12:37.447402+0000", 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036361199999999997 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-21T23:12:59.158 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:12:59.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:12:59.161 INFO:tasks.ceph.osd.1.smithi150.stderr:2024-09-21T23:12:59.157+0000 7f3ecf81a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26503) UID: 0 2024-09-21T23:12:59.260 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:59.257+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:59.302 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:59.302 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:59.302 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:59.302 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:59.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:12:59.362 INFO:tasks.ceph.osd.3.smithi150.stderr:2024-09-21T23:12:59.361+0000 7f6b835b0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26507) UID: 0 2024-09-21T23:12:59.434 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:59.434 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:59.434 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:59.434 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:59.434 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:59.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:12:59.462 INFO:tasks.ceph.osd.0.smithi150.stderr:2024-09-21T23:12:59.461+0000 7f0d169a5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26501) UID: 0 2024-09-21T23:12:59.563 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:59.561+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:59.573 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:59.573 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:12:59.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64443 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74716327900001, 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052759841000000002, 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.574 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64443 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823072+0000", 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823076+0000", 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7440000000000001e-06 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823086+0000", 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5759999999999995e-06 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823093+0000", 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875363+0000", 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052269950000000003 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875527+0000", 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016404399999999999 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875832+0000", 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030458500000000001 2024-09-21T23:12:59.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64445 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.747092271, 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053854105999999999, 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64445 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823143+0000", 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823147+0000", 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000002e-06 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823156+0000", 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4730000000000006e-06 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823167+0000", 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1046000000000001e-05 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875921+0000", 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052754323999999998 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876140+0000", 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000218648 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876997+0000", 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085674199999999996 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.583 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64455 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74702798, 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054260463000000002, 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.584 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64455 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823207+0000", 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823208+0000", 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823211+0000", 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1329999999999999e-06 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823220+0000", 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5670000000000001e-06 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823228+0000", 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877079+0000", 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053851547 2024-09-21T23:12:59.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877193+0000", 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114071 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877467+0000", 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027397899999999999 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.588 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64456 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74699748099999, 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875462, 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.589 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64456 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823237+0000", 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823238+0000", 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823241+0000", 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0769999999999999e-06 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823251+0000", 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6129999999999993e-06 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823261+0000", 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023e-05 2024-09-21T23:12:59.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877568+0000", 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054306902999999997 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877767+0000", 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019934999999999999 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878113+0000", 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000345886 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64459 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74684949500001, 2024-09-21T23:12:59.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055465686, 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64459 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823385+0000", 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823387+0000", 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T23:12:59.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823392+0000", 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6400000000000002e-06 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823409+0000", 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6679000000000001e-05 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823423+0000", 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4076999999999999e-05 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054787014000000002 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878449+0000", 2024-09-21T23:12:59.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023951699999999999 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878851+0000", 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040172599999999998 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64460 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.599 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74679299, 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052719981999999999, 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64460 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823442+0000", 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823443+0000", 2024-09-21T23:12:59.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.066e-06 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823449+0000", 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7570000000000001e-06 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823466+0000", 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7176e-05 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823479+0000", 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3315999999999999e-05 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875669+0000", 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052190126000000003 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.875889+0000", 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219707 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876162+0000", 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027283400000000001 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.604 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64461 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746738415, 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056014765000000001, 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64461 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823497+0000", 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823498+0000", 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823503+0000", 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2360000000000004e-06 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823521+0000", 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7952999999999999e-05 2024-09-21T23:12:59.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823535+0000", 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.434e-05 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878977+0000", 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055442157999999998 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879190+0000", 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021231900000000001 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879511+0000", 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032176200000000001 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64462 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746683197, 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053305847000000003, 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64462 2024-09-21T23:12:59.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823552+0000", 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823553+0000", 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.234e-06 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823558+0000", 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1599999999999997e-06 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823576+0000", 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7526000000000001e-05 2024-09-21T23:12:59.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823590+0000", 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3839000000000001e-05 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876411+0000", 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052821223 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876611+0000", 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020018 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876858+0000", 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024668500000000001 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.614 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64471 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74662856800001, 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056437001000000001, 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64471 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.615 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823606+0000", 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823607+0000", 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823613+0000", 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1399999999999999e-06 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823634+0000", 2024-09-21T23:12:59.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1495999999999999e-05 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823647+0000", 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3436e-05 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879578+0000", 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055930224000000001 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879783+0000", 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205157 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880043+0000", 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000260518 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64472 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746509418, 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056890108000000002, 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.620 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64472 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823726+0000", 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823727+0000", 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.353e-06 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823745+0000", 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.859e-05 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823767+0000", 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1072000000000001e-05 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823782+0000", 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5586999999999998e-05 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880127+0000", 2024-09-21T23:12:59.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056345336000000003 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880329+0000", 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000201669 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880616+0000", 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028650099999999999 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.624 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64473 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746431984, 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057235077000000002, 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64473 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823803+0000", 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823804+0000", 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2240000000000001e-06 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823810+0000", 2024-09-21T23:12:59.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823834+0000", 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4494999999999998e-05 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823849+0000", 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4843e-05 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880700+0000", 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056851143999999999 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880867+0000", 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016638699999999999 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881038+0000", 2024-09-21T23:12:59.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000171254 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64474 3.16 3:6b390c1e:::rbd_data.1149904dce8.0000000000000001:21 [read 1556480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746366362, 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057503552999999999, 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.630 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64474 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823869+0000", 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823870+0000", 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260000000000001e-06 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823876+0000", 2024-09-21T23:12:59.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823899+0000", 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842999999999999e-05 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823914+0000", 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4613000000000001e-05 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881089+0000", 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057174621000000002 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881208+0000", 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011890799999999999 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.881372+0000", 2024-09-21T23:12:59.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016456199999999999 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64480 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746302034, 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053464835000000002, 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.635 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64480 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823933+0000", 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823934+0000", 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823939+0000", 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.417e-06 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823958+0000", 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8756000000000001e-05 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823976+0000", 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8139e-05 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.876943+0000", 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052966646999999999 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877164+0000", 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022067899999999999 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877398+0000", 2024-09-21T23:12:59.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023413399999999999 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64482 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746240946, 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053983313999999998, 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.640 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64482 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823994+0000", 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.823995+0000", 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0300000000000001e-06 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824001+0000", 2024-09-21T23:12:59.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5430000000000001e-06 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824021+0000", 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0360000000000002e-05 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824036+0000", 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4732000000000001e-05 2024-09-21T23:12:59.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877481+0000", 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053445316 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877604+0000", 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012348200000000001 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.877977+0000", 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000372851 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64484 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746180481, 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054370072999999998, 2024-09-21T23:12:59.645 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64484 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:59.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824054+0000", 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824056+0000", 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.15e-06 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824061+0000", 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000003e-06 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824084+0000", 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2555999999999999e-05 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824097+0000", 2024-09-21T23:12:59.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3910000000000001e-05 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878070+0000", 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053973056999999998 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878210+0000", 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013976099999999999 2024-09-21T23:12:59.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878425+0000", 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000214296 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64486 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:59.650 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74611628300001, 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054948166999999999, 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64486 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824119+0000", 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824120+0000", 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.13e-06 2024-09-21T23:12:59.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824125+0000", 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6869999999999999e-06 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824145+0000", 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9298999999999999e-05 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824160+0000", 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5143000000000001e-05 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878481+0000", 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054320975000000001 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.878696+0000", 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021510999999999999 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879067+0000", 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370823 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.655 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64487 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74604227499999, 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055195978999999999, 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64487 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.656 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824193+0000", 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.37e-07 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824198+0000", 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2100000000000003e-06 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824209+0000", 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1836e-05 2024-09-21T23:12:59.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824219+0000", 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4599999999999992e-06 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879120+0000", 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054901158999999998 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879229+0000", 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109227 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879389+0000", 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015935 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64488 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.746004849, 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055624306999999998, 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64488 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.661 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824230+0000", 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824231+0000", 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7299999999999995e-07 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824234+0000", 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2629999999999999e-06 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824247+0000", 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3166000000000001e-05 2024-09-21T23:12:59.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824260+0000", 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2624e-05 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879437+0000", 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055176959999999997 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879566+0000", 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012914 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879854+0000", 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028848099999999999 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.665 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64489 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.745958327, 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056025048000000001, 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64489 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824277+0000", 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824278+0000", 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4890000000000001e-06 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824285+0000", 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824302+0000", 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7513e-05 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824317+0000", 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4113e-05 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.879924+0000", 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055607898000000003 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880042+0000", 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117484 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880302+0000", 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025973400000000002 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.670 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:64490 3.d 3:b1436818:::rbd_data.1149904dce8.0000000000000003:21 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.74589883199999, 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056373181000000001, 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 64490 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:12:59.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824336+0000", 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:12:59.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824337+0000", 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824343+0000", 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1759999999999998e-06 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824362+0000", 2024-09-21T23:12:59.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8499999999999999e-05 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.824377+0000", 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4919000000000001e-05 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880353+0000", 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055976271000000001 2024-09-21T23:12:59.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880487+0000", 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341199999999999 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:12.880709+0000", 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022276300000000001 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.676 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:12:59.676 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:12:59.676 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:59.676 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:12:59.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T23:12:59.678 INFO:tasks.ceph.osd.2.smithi150.stderr:2024-09-21T23:12:59.677+0000 7ff8504a4640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26505) UID: 0 2024-09-21T23:12:59.767 INFO:teuthology.orchestra.run.smithi150.stdout:All operations completed A-OK! 2024-09-21T23:12:59.778 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T23:12:59.775+0000 7f0e506a9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26596) UID: 0 2024-09-21T23:12:59.810 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:59.810 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:59.810 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:12:59.810 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:12:59.815 DEBUG:teuthology.parallel:result is None 2024-09-21T23:12:59.815 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-21T23:12:59.823 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T23:12:59.830 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T23:12:59.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T23:12:59.990 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:12:59.991 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:12:59.991 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:12:59.991 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:12:59.991 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:00.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T23:13:00.125 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:38083 3.11 3:8e5102db:::rbd_data.1149ed0ffb75.0000000000000000:head [stat,write 14664542~15522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 141.39487031100001, 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.998998202, 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.126 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 38083 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727303+0000", 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727305+0000", 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727335+0000", 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9530999999999999e-05 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727368+0000", 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2527999999999998e-05 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727383+0000", 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727530+0000", 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014656000000000001 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:38.727546+0000", 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6172999999999999e-05 2024-09-21T23:13:00.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720215+0000", 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.99266938199999999 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720456+0000", 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000240399 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 7", 2024-09-21T23:13:00.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.720695+0000", 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023877499999999999 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.722555+0000", 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001860631 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726090+0000", 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035344679999999998 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726179+0000", 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8896999999999999e-05 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:39.726301+0000", 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122347 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77707 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3456000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747556818000007, 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048418451000000001, 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77707 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374616+0000", 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374617+0000", 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374623+0000", 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4829999999999996e-06 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374631+0000", 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9300000000000003e-06 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374637+0000", 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9789999999999998e-06 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422848+0000", 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048210305000000002 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.422932+0000", 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4509e-05 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423035+0000", 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102557 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77708 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3467264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:13:00.138 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747524143000007, 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048584522999999998, 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77708 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374649+0000", 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2500000000000005e-07 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374652+0000", 2024-09-21T23:13:00.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374659+0000", 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374664+0000", 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.288e-06 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423068+0000", 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048403795999999999 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423131+0000", 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2491999999999997e-05 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423233+0000", 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T23:13:00.142 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77709 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3478528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747500807999998, 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048764808, 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.143 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77709 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374672+0000", 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374673+0000", 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374675+0000", 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3599999999999999e-06 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374683+0000", 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:13:00.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374689+0000", 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4960000000000004e-06 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423265+0000", 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048575989 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423348+0000", 2024-09-21T23:13:00.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2837000000000007e-05 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423437+0000", 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9387000000000003e-05 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77710 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3489792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747468243, 2024-09-21T23:13:00.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048905223999999997, 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77710 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374705+0000", 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5300000000000004e-07 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374708+0000", 2024-09-21T23:13:00.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8030000000000001e-06 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374720+0000", 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1752e-05 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374726+0000", 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8690000000000001e-06 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423471+0000", 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048744846000000001 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423526+0000", 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5287000000000001e-05 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423610+0000", 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4114000000000001e-05 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.151 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77711 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3501056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747380621000005, 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049013913999999999, 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.152 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77711 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374792+0000", 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374793+0000", 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.850000000000001e-07 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374798+0000", 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1690000000000002e-06 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374808+0000", 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.01e-05 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374815+0000", 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9290000000000001e-06 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423641+0000", 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048826256999999998 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423718+0000", 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.695e-05 2024-09-21T23:13:00.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423806+0000", 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8523999999999995e-05 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77712 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3512320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747261874000003, 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049066103999999999, 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.156 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77712 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374911+0000", 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374912+0000", 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374916+0000", 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.732e-06 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374922+0000", 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6120000000000004e-06 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374927+0000", 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999996e-06 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423833+0000", 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048906312 2024-09-21T23:13:00.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423899+0000", 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6625999999999995e-05 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.423977+0000", 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7787999999999995e-05 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.160 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77713 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3523584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747237108999997, 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049240994000000003, 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77713 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374936+0000", 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:13:00.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374938+0000", 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9709999999999998e-06 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374942+0000", 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1160000000000001e-06 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374946+0000", 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3689999999999998e-06 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424007+0000", 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049061057999999998 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424088+0000", 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1179000000000003e-05 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424177+0000", 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8888999999999995e-05 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77714 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3534848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747215961999999, 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049411935999999997, 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.165 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77714 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374957+0000", 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374959+0000", 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.781e-06 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374967+0000", 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.396e-06 2024-09-21T23:13:00.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.374970+0000", 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.393e-06 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424208+0000", 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049238482 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424285+0000", 2024-09-21T23:13:00.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6415999999999995e-05 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424369+0000", 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4136999999999998e-05 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77715 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3546112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:13:00.169 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747158264999996, 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049516945, 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77715 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:13:00.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375015+0000", 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375017+0000", 2024-09-21T23:13:00.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.761e-06 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375022+0000", 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375026+0000", 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2989999999999998e-06 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424399+0000", 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049373495000000003 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424476+0000", 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6367999999999998e-05 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424532+0000", 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5974999999999999e-05 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.173 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77716 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3557376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747123384000005, 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049633763999999997, 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.174 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77716 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375050+0000", 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375053+0000", 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375061+0000", 2024-09-21T23:13:00.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375067+0000", 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.575e-06 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424548+0000", 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049480967000000001 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424597+0000", 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9777000000000001e-05 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424683+0000", 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6095999999999997e-05 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77717 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3568640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.178 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747097858999993, 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049819898000000001, 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77717 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375075+0000", 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375076+0000", 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000005e-07 2024-09-21T23:13:00.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375078+0000", 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.773e-06 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375086+0000", 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0890000000000002e-06 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375089+0000", 2024-09-21T23:13:00.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3280000000000002e-06 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424714+0000", 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049625391999999997 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424791+0000", 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7102999999999997e-05 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424895+0000", 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103674 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77718 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3579904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.747026672999993, 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049971020999999997, 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.183 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77718 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375146+0000", 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375147+0000", 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375149+0000", 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375156+0000", 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9160000000000004e-06 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375159+0000", 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.424930+0000", 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049771229 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425018+0000", 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7557000000000002e-05 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425117+0000", 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9574999999999994e-05 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77719 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3591168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.187 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.746990827000005, 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050145220999999997, 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77719 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375182+0000", 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375183+0000", 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5600000000000001e-07 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375184+0000", 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375191+0000", 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375194+0000", 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4379999999999999e-06 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425148+0000", 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049954158999999998 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425227+0000", 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425327+0000", 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010019200000000001 2024-09-21T23:13:00.191 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77720 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3602432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.746973733999994, 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050347276000000003, 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.192 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77720 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375199+0000", 2024-09-21T23:13:00.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375200+0000", 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5100000000000001e-07 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375201+0000", 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5400000000000001e-06 2024-09-21T23:13:00.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375206+0000", 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375209+0000", 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8270000000000002e-06 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425360+0000", 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050151188999999999 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425444+0000", 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3673999999999999e-05 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425547+0000", 2024-09-21T23:13:00.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102624 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77721 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3613696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.746957722999994, 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050504034000000003, 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.197 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77721 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375215+0000", 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375216+0000", 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.53e-07 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375217+0000", 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4279999999999999e-06 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375221+0000", 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375224+0000", 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7460000000000001e-06 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425579+0000", 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050355577999999998 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425664+0000", 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4623999999999995e-05 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425719+0000", 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5609999999999998e-05 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.201 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77722 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3624960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.746900475000004, 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050601711000000001, 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77722 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375273+0000", 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.75e-07 2024-09-21T23:13:00.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375276+0000", 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3869999999999999e-06 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375283+0000", 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375289+0000", 2024-09-21T23:13:00.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8449999999999999e-06 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425736+0000", 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050446458 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425786+0000", 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9982999999999999e-05 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425874+0000", 2024-09-21T23:13:00.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8462999999999996e-05 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77723 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3636224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.746819110000004, 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050735871000000002, 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.206 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77723 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375354+0000", 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375355+0000", 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375358+0000", 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375366+0000", 2024-09-21T23:13:00.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3059999999999998e-06 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375373+0000", 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0820000000000004e-06 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425904+0000", 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050531833999999998 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.425988+0000", 2024-09-21T23:13:00.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4031999999999998e-05 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426090+0000", 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101345 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77724 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3647488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:13:00.210 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.746787560000001, 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050910417999999999, 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77724 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:13:00.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375385+0000", 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375386+0000", 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375389+0000", 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375395+0000", 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7000000000000002e-06 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375401+0000", 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6439999999999997e-06 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426117+0000", 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050715936000000003 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426195+0000", 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8153999999999997e-05 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426296+0000", 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100828 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.214 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:77725 3.a 3:533f2c0c:::rbd_data.11498e015c32.0000000000000006:27 [read 3658752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 97.746640662999994, 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050978398000000001, 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 77725 2024-09-21T23:13:00.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375532+0000", 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375533+0000", 2024-09-21T23:13:00.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375538+0000", 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7169999999999999e-06 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375549+0000", 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1345000000000001e-05 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.375557+0000", 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0059999999999992e-06 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426328+0000", 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050770468999999999 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426409+0000", 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0702000000000001e-05 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:22.426511+0000", 2024-09-21T23:13:00.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102229 2024-09-21T23:13:00.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.219 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:00.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T23:13:00.347 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:13:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:13:00.347 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:13:00.347 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:00.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T23:13:00.400 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:13:00.487 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:13:00.487 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:13:00.487 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:13:00.487 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:13:00.487 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:00.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4565 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.44453507899999, 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054875341000000001, 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.632 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4565 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183794+0000", 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183795+0000", 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183820+0000", 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.442e-05 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183851+0000", 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1208999999999999e-05 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183866+0000", 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5387e-05 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236092+0000", 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052225768999999998 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.236225+0000", 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278300000000001 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238669+0000", 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002444543 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4566 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.636 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.44443199299999, 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054857845000000002, 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4566 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:13:00.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183897+0000", 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183899+0000", 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183911+0000", 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2094e-05 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183931+0000", 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0023000000000001e-05 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183946+0000", 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4800000000000001e-05 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238736+0000", 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054789916000000001 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238755+0000", 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9516000000000001e-05 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:4567 3.f 3.e11c92ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.44436464399999, 2024-09-21T23:13:00.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054837787999999998, 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 4567 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183965+0000", 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183966+0000", 2024-09-21T23:13:00.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183972+0000", 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1630000000000001e-06 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.183990+0000", 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8257000000000002e-05 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.184004+0000", 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4222999999999999e-05 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238785+0000", 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054780415999999998 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.238802+0000", 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7659000000000002e-05 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5748 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1277952~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.369177708, 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053513006000000002, 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5748 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259152+0000", 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:13:00.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259155+0000", 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8930000000000001e-06 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259162+0000", 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5800000000000003e-06 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259168+0000", 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5029999999999997e-06 2024-09-21T23:13:00.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312280+0000", 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053112390000000002 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312325+0000", 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4406000000000001e-05 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312665+0000", 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033979099999999999 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5749 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36912194000001, 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053963610000000002, 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.649 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5749 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259207+0000", 2024-09-21T23:13:00.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259208+0000", 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.85e-07 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259212+0000", 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:13:00.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259221+0000", 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4199999999999996e-06 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259229+0000", 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6129999999999998e-06 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312684+0000", 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053455313999999997 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.312725+0000", 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1304999999999998e-05 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313171+0000", 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044565799999999998 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.653 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5750 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36909040099999, 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054196639999999997, 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5750 2024-09-21T23:13:00.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259239+0000", 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259240+0000", 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259247+0000", 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9869999999999999e-06 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259256+0000", 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259262+0000", 2024-09-21T23:13:00.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T23:13:00.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313232+0000", 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053969930999999999 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313312+0000", 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9883999999999996e-05 2024-09-21T23:13:00.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313435+0000", 2024-09-21T23:13:00.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123251 2024-09-21T23:13:00.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.659 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.659 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5751 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36905765200001, 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054732696999999997, 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.660 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5751 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:13:00.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:13:00.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259272+0000", 2024-09-21T23:13:00.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:13:00.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259275+0000", 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259284+0000", 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259290+0000", 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313493+0000", 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054202673 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.313633+0000", 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140295 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314004+0000", 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037106900000000002 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5752 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.369030759, 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055114267000000002, 2024-09-21T23:13:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5752 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259298+0000", 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259299+0000", 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259302+0000", 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259310+0000", 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2360000000000004e-06 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259315+0000", 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6210000000000001e-06 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314037+0000", 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054721277999999998 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314094+0000", 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7398000000000002e-05 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314413+0000", 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00031857000000000001 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.670 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5753 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 3571712~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36900548599999, 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055299354000000002, 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5753 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259324+0000", 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2499999999999995e-07 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259327+0000", 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7389999999999999e-06 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259337+0000", 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000004e-06 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259346+0000", 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8179999999999993e-06 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314461+0000", 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055115098000000001 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314537+0000", 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6281999999999999e-05 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314623+0000", 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5953999999999997e-05 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.674 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5754 3.f 3:f7493887:::rbd_data.114912f5d0f.0000000000000001:7 [read 4030464~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36897435899999, 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056283400999999997, 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5754 2024-09-21T23:13:00.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259355+0000", 2024-09-21T23:13:00.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259358+0000", 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259366+0000", 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8769999999999993e-06 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259381+0000", 2024-09-21T23:13:00.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4646e-05 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314668+0000", 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055286821 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.314737+0000", 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9327999999999998e-05 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315638+0000", 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00090120100000000002 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5882 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 1015808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36833100300001, 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055798287000000002, 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5882 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259998+0000", 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.259999+0000", 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.37e-07 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260001+0000", 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260010+0000", 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.260016+0000", 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4570000000000002e-06 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315671+0000", 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.055655234999999997 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315730+0000", 2024-09-21T23:13:00.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8349e-05 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.315797+0000", 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6685000000000003e-05 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5946 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 1736704~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36312359300001, 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053387174000000003, 2024-09-21T23:13:00.683 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5946 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265206+0000", 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265216+0000", 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7929999999999992e-06 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265221+0000", 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7190000000000001e-06 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265224+0000", 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7409999999999998e-06 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318440+0000", 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053215239999999997 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318518+0000", 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8792e-05 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318593+0000", 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4450999999999996e-05 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5947 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2195456~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.363099307, 2024-09-21T23:13:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053532030000000001, 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5947 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265230+0000", 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3099999999999998e-07 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265233+0000", 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.305e-06 2024-09-21T23:13:00.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265237+0000", 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6099999999999999e-06 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265242+0000", 2024-09-21T23:13:00.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1660000000000002e-06 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318637+0000", 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053394876000000001 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318693+0000", 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5328999999999998e-05 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318762+0000", 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9313000000000005e-05 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5948 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 2654208~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36308163999999, 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053838496, 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.692 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5948 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265248+0000", 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7700000000000001e-07 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265249+0000", 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.604e-06 2024-09-21T23:13:00.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265253+0000", 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8970000000000001e-06 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265257+0000", 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.416e-06 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318812+0000", 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053555347000000003 2024-09-21T23:13:00.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.318940+0000", 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012776199999999999 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319086+0000", 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146193 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5949 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3112960~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.696 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36305805500001, 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053986885999999998, 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5949 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265271+0000", 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.72e-07 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265273+0000", 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8980000000000001e-06 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265278+0000", 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5469999999999998e-06 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265282+0000", 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319131+0000", 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053849468999999997 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319187+0000", 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6407999999999999e-05 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319258+0000", 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0601999999999999e-05 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5950 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 3571712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36304204000001, 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054255824000000001, 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5950 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265287+0000", 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4900000000000002e-07 2024-09-21T23:13:00.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265289+0000", 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5719999999999999e-06 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265293+0000", 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8600000000000003e-06 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265297+0000", 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7469999999999999e-06 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319305+0000", 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054007942000000003 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319429+0000", 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012398199999999999 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319543+0000", 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114472 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5951 3.f 3:f61bd46c:::rbd_data.114912f5d0f.0000000000000016:7 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36302820899999, 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054505267000000003, 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5951 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265301+0000", 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265303+0000", 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5689999999999999e-06 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265307+0000", 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7979999999999999e-06 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265310+0000", 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3699999999999999e-06 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319582+0000", 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054272048000000003 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319634+0000", 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1823999999999997e-05 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319806+0000", 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017239600000000001 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5963 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.36299684799999, 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054771256999999997, 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5963 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265332+0000", 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265333+0000", 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265334+0000", 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.691e-06 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265339+0000", 2024-09-21T23:13:00.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1169999999999997e-06 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265342+0000", 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319869+0000", 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054527385999999997 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.319993+0000", 2024-09-21T23:13:00.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124079 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320104+0000", 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110522 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:5967 3.f 3:f1ef97e1:::rbd_data.114952d23f01.000000000000000c:6 [read 491520~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 173.362928426, 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054938714, 2024-09-21T23:13:00.713 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 5967 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265401+0000", 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265402+0000", 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1500000000000004e-07 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265404+0000", 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6220000000000001e-06 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265409+0000", 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7489999999999998e-06 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.265415+0000", 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320149+0000", 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054734620999999997 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320242+0000", 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2814999999999998e-05 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:10:07.320340+0000", 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7452000000000001e-05 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123547 3.13 3:c9e2c0b1:::rbd_data.11495bdc1465.000000000000000d:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:13:00.717 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 33.886240936999997, 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88065156200000005, 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:00.718 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123547 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742088+0000", 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742119+0000", 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0244e-05 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742127+0000", 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6100000000000006e-06 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742154+0000", 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6730999999999999e-05 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742165+0000", 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1419e-05 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742241+0000", 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5562999999999995e-05 2024-09-21T23:13:00.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.742251+0000", 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0017000000000001e-05 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619014+0000", 2024-09-21T23:13:00.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87676288899999999 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619157+0000", 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143549 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 4", 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.619280+0000", 2024-09-21T23:13:00.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012234100000000001 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621009+0000", 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001729511 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:13:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622618+0000", 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001609262 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622687+0000", 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8813000000000007e-05 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:00.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.622740+0000", 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2612999999999999e-05 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:00.727 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:00.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:13:00.855 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:13:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:13:00.856 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T23:13:00.856 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:00.870 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:13:00.992 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:13:00.992 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T23:13:00.992 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T23:13:00.992 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T23:13:00.992 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:01.006 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:13:01.127 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T23:13:01.127 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T23:13:01.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T23:13:01.127 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T23:13:01.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.127 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:61281 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:13:01.127 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 119.733651348, 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.975975814, 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 61281 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390204+0000", 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390206+0000", 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9929999999999998e-06 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390216+0000", 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390249+0000", 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2895000000000003e-05 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390266+0000", 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390362+0000", 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5705999999999995e-05 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:01.390380+0000", 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7473999999999999e-05 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366159+0000", 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.97577956200000004 2024-09-21T23:13:01.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:02.366180+0000", 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0971e-05 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71502 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7212032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263686479, 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069728088999999993, 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.132 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71502 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860169+0000", 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860170+0000", 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860172+0000", 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.384e-06 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860179+0000", 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.126e-06 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860185+0000", 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6720000000000001e-06 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929705+0000", 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069520338000000001 2024-09-21T23:13:01.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929797+0000", 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1947000000000006e-05 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929897+0000", 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010021 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71503 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7223296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.136 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263663324000007, 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069902776, 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71503 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860192+0000", 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860193+0000", 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1600000000000002e-07 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860195+0000", 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860202+0000", 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9430000000000003e-06 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860207+0000", 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2159999999999997e-06 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.929943+0000", 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069735695 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930029+0000", 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6304000000000006e-05 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930095+0000", 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5585000000000003e-05 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.140 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71504 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7234560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263641106999998, 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070425507999999998, 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.141 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71504 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860215+0000", 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3200000000000001e-07 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860217+0000", 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0789999999999999e-06 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860224+0000", 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860229+0000", 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8940000000000001e-06 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930468+0000", 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070239290999999995 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930535+0000", 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6335999999999996e-05 2024-09-21T23:13:01.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930640+0000", 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010547 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71505 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7245824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263620496000001, 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587797999999993, 2024-09-21T23:13:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71505 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860235+0000", 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1300000000000001e-07 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860237+0000", 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0020000000000001e-06 2024-09-21T23:13:01.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860244+0000", 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2990000000000002e-06 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860249+0000", 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9189999999999998e-06 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930672+0000", 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070423214999999997 2024-09-21T23:13:01.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930749+0000", 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6643999999999993e-05 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930823+0000", 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4406000000000006e-05 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71506 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7257088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263600871999998, 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070765725000000002, 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71506 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860255+0000", 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5999999999999999e-07 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860257+0000", 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.886e-06 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860263+0000", 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860268+0000", 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0039999999999999e-06 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930856+0000", 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070587467000000001 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.930928+0000", 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2473999999999998e-05 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931020+0000", 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2057000000000003e-05 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71507 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7268352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263579934999996, 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070969913999999995, 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71507 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860276+0000", 2024-09-21T23:13:01.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T23:13:01.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860279+0000", 2024-09-21T23:13:01.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0520000000000002e-06 2024-09-21T23:13:01.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860285+0000", 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.336e-06 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860291+0000", 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.694e-06 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931064+0000", 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070773338000000005 2024-09-21T23:13:01.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931137+0000", 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2836000000000005e-05 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931246+0000", 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108299 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.159 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71508 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7279616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263557972000001, 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071168304000000002, 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71508 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860298+0000", 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:13:01.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860300+0000", 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0549999999999998e-06 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860306+0000", 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860311+0000", 2024-09-21T23:13:01.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6789999999999996e-06 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931284+0000", 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070972434000000001 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931385+0000", 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101779 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931466+0000", 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0616000000000002e-05 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71509 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7290880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263538548, 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071298687999999999, 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.164 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71509 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860317+0000", 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860319+0000", 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860326+0000", 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4150000000000004e-06 2024-09-21T23:13:01.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860331+0000", 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7600000000000002e-06 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931504+0000", 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071173599000000004 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931554+0000", 2024-09-21T23:13:01.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0130000000000003e-05 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931616+0000", 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71510 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7302144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263519407000004, 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071503908000000005, 2024-09-21T23:13:01.168 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71510 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860336+0000", 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860337+0000", 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860344+0000", 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:13:01.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860351+0000", 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7349999999999999e-06 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860356+0000", 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931648+0000", 2024-09-21T23:13:01.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071291820000000006 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931743+0000", 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5101999999999998e-05 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931840+0000", 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7400999999999998e-05 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71511 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7313408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263492956999997, 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071640652999999999, 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71511 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860363+0000", 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3299999999999998e-07 2024-09-21T23:13:01.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860365+0000", 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860371+0000", 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4359999999999999e-06 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860376+0000", 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.814e-06 2024-09-21T23:13:01.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931882+0000", 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071505767999999997 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.931942+0000", 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9858000000000002e-05 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932003+0000", 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1401000000000004e-05 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71512 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7324672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263473437000002, 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071778649, 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.177 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71512 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860382+0000", 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860383+0000", 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860384+0000", 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.764e-06 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860390+0000", 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:13:01.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860395+0000", 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7720000000000002e-06 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932032+0000", 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071637075999999994 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932104+0000", 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2127999999999998e-05 2024-09-21T23:13:01.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932161+0000", 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6397000000000002e-05 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71513 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 7335936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263450630999998, 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071894477999999998, 2024-09-21T23:13:01.181 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71513 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860405+0000", 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860407+0000", 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8679999999999999e-06 2024-09-21T23:13:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860414+0000", 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4289999999999998e-06 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860419+0000", 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932179+0000", 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071759559000000001 2024-09-21T23:13:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932221+0000", 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2571000000000003e-05 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932299+0000", 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8102000000000005e-05 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71626 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6818816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263430158999995, 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072052277999999997, 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71626 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860425+0000", 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860426+0000", 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860428+0000", 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9690000000000001e-06 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860435+0000", 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7680000000000003e-06 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860439+0000", 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8709999999999996e-06 2024-09-21T23:13:01.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932338+0000", 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071898218999999999 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932397+0000", 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9361999999999998e-05 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932478+0000", 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0734000000000003e-05 2024-09-21T23:13:01.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71627 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6830080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263354394000004, 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072261401000000003, 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.190 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71627 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860501+0000", 2024-09-21T23:13:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860502+0000", 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860505+0000", 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7030000000000002e-06 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860512+0000", 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3440000000000003e-06 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860517+0000", 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932530+0000", 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072012127999999995 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932656+0000", 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012687399999999999 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932763+0000", 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010617599999999999 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71628 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6841344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.194 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.263330722000006, 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072541103999999995, 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71628 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.860525+0000", 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865096+0000", 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045708149999999998 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865102+0000", 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0809999999999999e-06 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865113+0000", 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1099e-05 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865122+0000", 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9220000000000003e-06 2024-09-21T23:13:01.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932824+0000", 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.067702410000000005 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.932929+0000", 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010447500000000001 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.933066+0000", 2024-09-21T23:13:01.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013730200000000001 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71638 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6953984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.258415550999999, 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632171000000007, 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.199 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71638 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:13:01.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865440+0000", 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865443+0000", 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5229999999999998e-06 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865450+0000", 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1760000000000004e-06 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865456+0000", 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934928+0000", 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069471791000000005 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.934985+0000", 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935072+0000", 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7408000000000001e-05 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71639 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6965248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.203 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.258390825000006, 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069829904999999998, 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71639 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865465+0000", 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865468+0000", 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.785e-06 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865476+0000", 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865482+0000", 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:13:01.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935114+0000", 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069632406999999993 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935198+0000", 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3753999999999998e-05 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935295+0000", 2024-09-21T23:13:01.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6804000000000004e-05 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:71640 3.14 3:2d5d8213:::rbd_data.11498e015c32.0000000000000003:27 [read 6976512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 99.258366249999995, 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069982146999999995, 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.208 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 71640 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.865489+0000", 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.866997+0000", 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015074999999999999 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867002+0000", 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1619999999999999e-06 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867014+0000", 2024-09-21T23:13:01.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.221e-05 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.867022+0000", 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1440000000000003e-06 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935342+0000", 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068319243000000002 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935402+0000", 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0359000000000003e-05 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:11:21.935472+0000", 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9529000000000005e-05 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.4425.0:123558 3.9 3:93c04306:::rbd_data.11495bdc1465.0000000000000026:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:13:01.212 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 34.377416783999998, 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.88255402500000002, 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.4425", 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.150:0/602335799", 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123558 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746439+0000", 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746441+0000", 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0530000000000001e-06 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746450+0000", 2024-09-21T23:13:01.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6889999999999993e-06 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746471+0000", 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1005999999999998e-05 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746485+0000", 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746627+0000", 2024-09-21T23:13:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014169399999999999 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:26.746638+0000", 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1311000000000001e-05 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621095+0000", 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.87445676999999999 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621273+0000", 2024-09-21T23:13:01.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017780399999999999 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2", 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.621358+0000", 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5068999999999995e-05 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.623621+0000", 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022634690000000002 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:13:01.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628925+0000", 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053032009999999996 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628961+0000", 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5936000000000003e-05 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T23:12:27.628993+0000", 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2274000000000003e-05 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.218 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T23:13:01.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T23:13:01.219 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T23:13:01.605 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-21T23:13:01.605 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-21T23:13:01.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-21T23:13:01.918 INFO:tasks.ceph.mon.a.smithi150.stderr:2024-09-21T23:13:01.917+0000 7f4d03887640 -1 mon.a@0(leader).osd e179 definitely_dead 0 2024-09-21T23:13:02.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:13:02.626 INFO:teuthology.orchestra.run.smithi150.stderr:marked in osd.6. 2024-09-21T23:13:02.650 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-21T23:13:02.650 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T23:13:02.781 INFO:teuthology.orchestra.run.smithi150.stdout:mon.a: {} 2024-09-21T23:13:02.782 INFO:teuthology.orchestra.run.smithi150.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:13:02.794 INFO:teuthology.orchestra.run.smithi150.stdout:mon.b: {} 2024-09-21T23:13:02.794 INFO:teuthology.orchestra.run.smithi150.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:13:02.803 INFO:teuthology.orchestra.run.smithi150.stdout:mon.c: {} 2024-09-21T23:13:02.803 INFO:teuthology.orchestra.run.smithi150.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:13:02.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:13:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:13:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:noscrub is unset 2024-09-21T23:13:03.666 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:13:03.667 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T23:13:04.632 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T23:13:04.642 INFO:teuthology.orchestra.run.smithi150.stderr:nodeep-scrub is unset 2024-09-21T23:13:04.668 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:13:05.403 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:13:05.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T23:13:06.413 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T23:13:06.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:13:07.413 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:13:07.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T23:13:08.677 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T23:13:08.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:13:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:13:09.704 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T23:13:10.695 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T23:13:10.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:13:11.698 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:13:11.724 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T23:13:12.724 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T23:13:12.750 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T23:13:13.727 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T23:13:13.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-21T23:13:14.736 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-21T23:13:14.762 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T23:13:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T23:13:15.764 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-21T23:13:16.754 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-21T23:13:16.781 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:13:17.765 INFO:teuthology.orchestra.run.smithi150.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:13:17.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T23:13:18.777 INFO:teuthology.orchestra.run.smithi150.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T23:13:18.801 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T23:13:18.801 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T23:13:18.801 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T23:13:18.801 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T23:13:18.801 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T23:13:18.801 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:13:19.102 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:19.102 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":196,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:13:18.766401+0000","last_up_change":"2024-09-21T23:11:01.346212+0000","last_in_change":"2024-09-21T23:13:02.619106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:09:40.569134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:09:44.626156+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T23:09:53.888601+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":178,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6817","nonce":198356244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6819","nonce":198356244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6823","nonce":198356244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6821","nonce":198356244}]},"public_addr":"172.21.15.150:6817/198356244","cluster_addr":"172.21.15.150:6819/198356244","heartbeat_back_addr":"172.21.15.150:6823/198356244","heartbeat_front_addr":"172.21.15.150:6821/198356244","state":["exists","up"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6825","nonce":725319294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6827","nonce":725319294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6831","nonce":725319294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6829","nonce":725319294}]},"public_addr":"172.21.15.150:6825/725319294","cluster_addr":"172.21.15.150:6827/725319294","heartbeat_back_addr":"172.21.15.150:6831/725319294","heartbeat_front_addr":"172.21.15.150:6829/725319294","state":["exists","up"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6801","nonce":1956153436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6803","nonce":1956153436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6807","nonce":1956153436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6805","nonce":1956153436}]},"public_addr":"172.21.15.150:6801/1956153436","cluster_addr":"172.21.15.150:6803/1956153436","heartbeat_back_addr":"172.21.15.150:6807/1956153436","heartbeat_front_addr":"172.21.15.150:6805/1956153436","state":["exists","up"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6809","nonce":3049870482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6811","nonce":3049870482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6815","nonce":3049870482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6813","nonce":3049870482}]},"public_addr":"172.21.15.150:6809/3049870482","cluster_addr":"172.21.15.150:6811/3049870482","heartbeat_back_addr":"172.21.15.150:6815/3049870482","heartbeat_front_addr":"172.21.15.150:6813/3049870482","state":["exists","up"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":32,"up_from":69,"up_thru":178,"down_at":33,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6801","nonce":3379790213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6803","nonce":3379790213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6807","nonce":3379790213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6805","nonce":3379790213}]},"public_addr":"172.21.15.175:6801/3379790213","cluster_addr":"172.21.15.175:6803/3379790213","heartbeat_back_addr":"172.21.15.175:6807/3379790213","heartbeat_front_addr":"172.21.15.175:6805/3379790213","state":["exists","up"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6827","nonce":120048378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6829","nonce":120048378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6833","nonce":120048378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6831","nonce":120048378}]},"public_addr":"172.21.15.175:6827/120048378","cluster_addr":"172.21.15.175:6829/120048378","heartbeat_back_addr":"172.21.15.175:6833/120048378","heartbeat_front_addr":"172.21.15.175:6831/120048378","state":["exists","up"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6819","nonce":787648514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6821","nonce":787648514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6825","nonce":787648514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6823","nonce":787648514}]},"public_addr":"172.21.15.175:6819/787648514","cluster_addr":"172.21.15.175:6821/787648514","heartbeat_back_addr":"172.21.15.175:6825/787648514","heartbeat_front_addr":"172.21.15.175:6823/787648514","state":["exists","up"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6809","nonce":1424462696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6811","nonce":1424462696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6815","nonce":1424462696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6813","nonce":1424462696}]},"public_addr":"172.21.15.175:6809/1424462696","cluster_addr":"172.21.15.175:6811/1424462696","heartbeat_back_addr":"172.21.15.175:6815/1424462696","heartbeat_front_addr":"172.21.15.175:6813/1424462696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.004700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.958497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.988302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.993971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.945648+0000","dead_epoch":33},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.366302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.852613+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.713697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:13:19.121 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T23:13:19.121 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:13:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:19.417 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":196,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:13:18.766401+0000","last_up_change":"2024-09-21T23:11:01.346212+0000","last_in_change":"2024-09-21T23:13:02.619106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:09:40.569134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:09:44.626156+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T23:09:53.888601+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":178,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6817","nonce":198356244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6819","nonce":198356244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6823","nonce":198356244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6821","nonce":198356244}]},"public_addr":"172.21.15.150:6817/198356244","cluster_addr":"172.21.15.150:6819/198356244","heartbeat_back_addr":"172.21.15.150:6823/198356244","heartbeat_front_addr":"172.21.15.150:6821/198356244","state":["exists","up"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6825","nonce":725319294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6827","nonce":725319294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6831","nonce":725319294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6829","nonce":725319294}]},"public_addr":"172.21.15.150:6825/725319294","cluster_addr":"172.21.15.150:6827/725319294","heartbeat_back_addr":"172.21.15.150:6831/725319294","heartbeat_front_addr":"172.21.15.150:6829/725319294","state":["exists","up"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6801","nonce":1956153436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6803","nonce":1956153436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6807","nonce":1956153436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6805","nonce":1956153436}]},"public_addr":"172.21.15.150:6801/1956153436","cluster_addr":"172.21.15.150:6803/1956153436","heartbeat_back_addr":"172.21.15.150:6807/1956153436","heartbeat_front_addr":"172.21.15.150:6805/1956153436","state":["exists","up"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6809","nonce":3049870482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6811","nonce":3049870482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6815","nonce":3049870482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6813","nonce":3049870482}]},"public_addr":"172.21.15.150:6809/3049870482","cluster_addr":"172.21.15.150:6811/3049870482","heartbeat_back_addr":"172.21.15.150:6815/3049870482","heartbeat_front_addr":"172.21.15.150:6813/3049870482","state":["exists","up"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":32,"up_from":69,"up_thru":178,"down_at":33,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6801","nonce":3379790213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6803","nonce":3379790213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6807","nonce":3379790213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6805","nonce":3379790213}]},"public_addr":"172.21.15.175:6801/3379790213","cluster_addr":"172.21.15.175:6803/3379790213","heartbeat_back_addr":"172.21.15.175:6807/3379790213","heartbeat_front_addr":"172.21.15.175:6805/3379790213","state":["exists","up"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6827","nonce":120048378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6829","nonce":120048378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6833","nonce":120048378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6831","nonce":120048378}]},"public_addr":"172.21.15.175:6827/120048378","cluster_addr":"172.21.15.175:6829/120048378","heartbeat_back_addr":"172.21.15.175:6833/120048378","heartbeat_front_addr":"172.21.15.175:6831/120048378","state":["exists","up"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6819","nonce":787648514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6821","nonce":787648514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6825","nonce":787648514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6823","nonce":787648514}]},"public_addr":"172.21.15.175:6819/787648514","cluster_addr":"172.21.15.175:6821/787648514","heartbeat_back_addr":"172.21.15.175:6825/787648514","heartbeat_front_addr":"172.21.15.175:6823/787648514","state":["exists","up"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6809","nonce":1424462696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6811","nonce":1424462696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6815","nonce":1424462696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6813","nonce":1424462696}]},"public_addr":"172.21.15.175:6809/1424462696","cluster_addr":"172.21.15.175:6811/1424462696","heartbeat_back_addr":"172.21.15.175:6815/1424462696","heartbeat_front_addr":"172.21.15.175:6813/1424462696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.004700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.958497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.988302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.993971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.945648+0000","dead_epoch":33},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.366302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.852613+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.713697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:13:19.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T23:13:19.436 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T23:13:19.436 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T23:13:19.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T23:13:19.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T23:13:19.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T23:13:19.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T23:13:19.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T23:13:19.668 INFO:teuthology.orchestra.run.smithi150.stdout:55834574896 2024-09-21T23:13:19.668 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:13:19.702 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:19.702 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:13:19.712 INFO:teuthology.orchestra.run.smithi150.stdout:55834574897 2024-09-21T23:13:19.713 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T23:13:19.714 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:19.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:13:19.716 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:19.716 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:13:19.718 INFO:teuthology.orchestra.run.smithi150.stdout:296352743453 2024-09-21T23:13:19.718 INFO:teuthology.orchestra.run.smithi150.stdout:55834574897 2024-09-21T23:13:19.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:13:19.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T23:13:19.720 INFO:teuthology.orchestra.run.smithi150.stdout:55834574896 2024-09-21T23:13:19.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:13:20.175 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:20.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574895 for osd.3 2024-09-21T23:13:20.220 INFO:teuthology.orchestra.run.smithi150.stdout:55834574896 2024-09-21T23:13:20.223 INFO:teuthology.orchestra.run.smithi150.stdout:55834574894 2024-09-21T23:13:20.235 INFO:teuthology.orchestra.run.smithi150.stdout:55834574894 2024-09-21T23:13:20.245 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:20.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574897 got 55834574896 for osd.7 2024-09-21T23:13:20.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.5 2024-09-21T23:13:20.262 INFO:teuthology.orchestra.run.smithi150.stdout:296352743452 2024-09-21T23:13:20.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574895 for osd.1 2024-09-21T23:13:20.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.2 2024-09-21T23:13:20.279 INFO:teuthology.orchestra.run.smithi150.stdout:55834574896 2024-09-21T23:13:20.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 296352743453 got 296352743452 for osd.4 2024-09-21T23:13:20.282 INFO:teuthology.orchestra.run.smithi150.stdout:55834574894 2024-09-21T23:13:20.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574897 got 55834574896 for osd.6 2024-09-21T23:13:20.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.0 2024-09-21T23:13:21.199 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:13:21.254 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T23:13:21.256 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:13:21.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:13:21.270 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:13:21.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:13:21.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T23:13:21.302 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:13:21.719 INFO:teuthology.orchestra.run.smithi150.stdout:55834574896 2024-09-21T23:13:21.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574896 for osd.3 2024-09-21T23:13:21.745 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.758 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:21.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.5 2024-09-21T23:13:21.776 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.803 INFO:teuthology.orchestra.run.smithi150.stdout:55834574897 2024-09-21T23:13:21.824 INFO:teuthology.orchestra.run.smithi150.stdout:296352743454 2024-09-21T23:13:21.827 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574897 got 55834574897 for osd.7 2024-09-21T23:13:21.827 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.830 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:21.831 INFO:teuthology.orchestra.run.smithi150.stdout:55834574896 2024-09-21T23:13:21.842 INFO:teuthology.orchestra.run.smithi150.stdout:55834574897 2024-09-21T23:13:21.853 INFO:teuthology.orchestra.run.smithi150.stdout:55834574895 2024-09-21T23:13:21.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 296352743453 got 296352743454 for osd.4 2024-09-21T23:13:21.856 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574896 for osd.1 2024-09-21T23:13:21.859 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.2 2024-09-21T23:13:21.861 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574897 got 55834574897 for osd.6 2024-09-21T23:13:21.870 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.0 2024-09-21T23:13:21.874 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:21.874 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:13:21.874 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:13:22.192 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:22.193 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:13:22.209 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-21T23:13:20.547590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":728,"num_whiteouts":720,"num_read":186277,"num_read_kb":840874,"num_write":22576,"num_write_kb":402378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5305,"num_bytes_recovered":255588574,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21054,"ondisk_log_size":21054,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":390148,"kb_used_data":19660,"kb_used_omap":25,"kb_used_meta":370278,"kb_avail":754584572,"statfs":{"total":773094113280,"available":772694601728,"internally_reserved":0,"allocated":20131840,"data_stored":10373255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":379164719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.807217"},"pg_stats":[{"pgid":"3.1f","version":"178'580","reported_seq":4220,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894261+0000","last_change":"2024-09-21T23:13:05.425494+0000","last_active":"2024-09-21T23:13:18.894261+0000","last_peered":"2024-09-21T23:13:18.894261+0000","last_clean":"2024-09-21T23:13:18.894261+0000","last_became_active":"2024-09-21T23:13:05.423231+0000","last_became_peered":"2024-09-21T23:13:05.423231+0000","last_unstale":"2024-09-21T23:13:18.894261+0000","last_undegraded":"2024-09-21T23:13:18.894261+0000","last_fullsized":"2024-09-21T23:13:18.894261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:12:08.608124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087587719999999997,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6355,"num_read_kb":19114,"num_write":548,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"172'894","reported_seq":4765,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776370+0000","last_change":"2024-09-21T23:13:07.423842+0000","last_active":"2024-09-21T23:13:18.776370+0000","last_peered":"2024-09-21T23:13:18.776370+0000","last_clean":"2024-09-21T23:13:18.776370+0000","last_became_active":"2024-09-21T23:13:07.421837+0000","last_became_peered":"2024-09-21T23:13:07.421837+0000","last_unstale":"2024-09-21T23:13:18.776370+0000","last_undegraded":"2024-09-21T23:13:18.776370+0000","last_fullsized":"2024-09-21T23:13:18.776370+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:26.907455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001882955,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5630,"num_read_kb":41276,"num_write":1148,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'456","reported_seq":3457,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894172+0000","last_change":"2024-09-21T23:12:59.616455+0000","last_active":"2024-09-21T23:13:18.894172+0000","last_peered":"2024-09-21T23:13:18.894172+0000","last_clean":"2024-09-21T23:13:18.894172+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:13:18.894172+0000","last_undegraded":"2024-09-21T23:13:18.894172+0000","last_fullsized":"2024-09-21T23:13:18.894172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:55.115242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158018,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3362,"num_read_kb":13834,"num_write":377,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'500","reported_seq":5839,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894768+0000","last_change":"2024-09-21T23:12:59.618158+0000","last_active":"2024-09-21T23:13:18.894768+0000","last_peered":"2024-09-21T23:13:18.894768+0000","last_clean":"2024-09-21T23:13:18.894768+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:13:18.894768+0000","last_undegraded":"2024-09-21T23:13:18.894768+0000","last_fullsized":"2024-09-21T23:13:18.894768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:16.558115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037616780000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5378,"num_read_kb":22578,"num_write":533,"num_write_kb":12630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"172'618","reported_seq":4487,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894025+0000","last_change":"2024-09-21T23:13:05.423467+0000","last_active":"2024-09-21T23:13:18.894025+0000","last_peered":"2024-09-21T23:13:18.894025+0000","last_clean":"2024-09-21T23:13:18.894025+0000","last_became_active":"2024-09-21T23:13:05.421427+0000","last_became_peered":"2024-09-21T23:13:05.421427+0000","last_unstale":"2024-09-21T23:13:18.894025+0000","last_undegraded":"2024-09-21T23:13:18.894025+0000","last_fullsized":"2024-09-21T23:13:18.894025+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:42.645544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010738146,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5513,"num_read_kb":28043,"num_write":694,"num_write_kb":17938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'1323","reported_seq":7922,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772720+0000","last_change":"2024-09-21T23:12:59.615466+0000","last_active":"2024-09-21T23:13:18.772720+0000","last_peered":"2024-09-21T23:13:18.772720+0000","last_clean":"2024-09-21T23:13:18.772720+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:13:18.772720+0000","last_undegraded":"2024-09-21T23:13:18.772720+0000","last_fullsized":"2024-09-21T23:13:18.772720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:14.383756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046765799999999996,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":12289,"num_read_kb":29931,"num_write":1690,"num_write_kb":9650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"172'1009","reported_seq":3157,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893498+0000","last_change":"2024-09-21T23:12:59.622668+0000","last_active":"2024-09-21T23:13:18.893498+0000","last_peered":"2024-09-21T23:13:18.893498+0000","last_clean":"2024-09-21T23:13:18.893498+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:13:18.893498+0000","last_undegraded":"2024-09-21T23:13:18.893498+0000","last_fullsized":"2024-09-21T23:13:18.893498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:39.668516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006449682,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6326,"num_read_kb":21971,"num_write":1459,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"172'1057","reported_seq":5547,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775421+0000","last_change":"2024-09-21T23:12:59.613494+0000","last_active":"2024-09-21T23:13:18.775421+0000","last_peered":"2024-09-21T23:13:18.775421+0000","last_clean":"2024-09-21T23:13:18.775421+0000","last_became_active":"2024-09-21T23:12:37.444303+0000","last_became_peered":"2024-09-21T23:12:37.444303+0000","last_unstale":"2024-09-21T23:13:18.775421+0000","last_undegraded":"2024-09-21T23:13:18.775421+0000","last_fullsized":"2024-09-21T23:13:18.775421+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:05.804247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030213219999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8120,"num_read_kb":28946,"num_write":1438,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5999502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'652","reported_seq":5692,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894396+0000","last_change":"2024-09-21T23:13:05.425470+0000","last_active":"2024-09-21T23:13:18.894396+0000","last_peered":"2024-09-21T23:13:18.894396+0000","last_clean":"2024-09-21T23:13:18.894396+0000","last_became_active":"2024-09-21T23:13:05.423133+0000","last_became_peered":"2024-09-21T23:13:05.423133+0000","last_unstale":"2024-09-21T23:13:18.894396+0000","last_undegraded":"2024-09-21T23:13:18.894396+0000","last_fullsized":"2024-09-21T23:13:18.894396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:38.996927+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022352600000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":6037,"num_read_kb":23064,"num_write":602,"num_write_kb":12198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'521","reported_seq":4585,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894097+0000","last_change":"2024-09-21T23:12:59.621765+0000","last_active":"2024-09-21T23:13:18.894097+0000","last_peered":"2024-09-21T23:13:18.894097+0000","last_clean":"2024-09-21T23:13:18.894097+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:13:18.894097+0000","last_undegraded":"2024-09-21T23:13:18.894097+0000","last_fullsized":"2024-09-21T23:13:18.894097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:11.726440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066845790000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5732,"num_read_kb":29704,"num_write":420,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"178'561","reported_seq":6978,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894886+0000","last_change":"2024-09-21T23:12:59.624274+0000","last_active":"2024-09-21T23:13:18.894886+0000","last_peered":"2024-09-21T23:13:18.894886+0000","last_clean":"2024-09-21T23:13:18.894886+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:13:18.894886+0000","last_undegraded":"2024-09-21T23:13:18.894886+0000","last_fullsized":"2024-09-21T23:13:18.894886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:35.264209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048388729999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7517,"num_read_kb":36655,"num_write":459,"num_write_kb":15039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'703","reported_seq":5090,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773133+0000","last_change":"2024-09-21T23:12:59.613161+0000","last_active":"2024-09-21T23:13:18.773133+0000","last_peered":"2024-09-21T23:13:18.773133+0000","last_clean":"2024-09-21T23:13:18.773133+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:13:18.773133+0000","last_undegraded":"2024-09-21T23:13:18.773133+0000","last_fullsized":"2024-09-21T23:13:18.773133+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:56.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024929100000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5571,"num_read_kb":14116,"num_write":826,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895381+0000","last_change":"2024-09-21T23:13:03.807826+0000","last_active":"2024-09-21T23:13:18.895381+0000","last_peered":"2024-09-21T23:13:18.895381+0000","last_clean":"2024-09-21T23:13:18.895381+0000","last_became_active":"2024-09-21T23:13:03.806956+0000","last_became_peered":"2024-09-21T23:13:03.806956+0000","last_unstale":"2024-09-21T23:13:18.895381+0000","last_undegraded":"2024-09-21T23:13:18.895381+0000","last_fullsized":"2024-09-21T23:13:18.895381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:13.318790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'617","reported_seq":2796,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894136+0000","last_change":"2024-09-21T23:13:05.416619+0000","last_active":"2024-09-21T23:13:18.894136+0000","last_peered":"2024-09-21T23:13:18.894136+0000","last_clean":"2024-09-21T23:13:18.894136+0000","last_became_active":"2024-09-21T23:13:05.415035+0000","last_became_peered":"2024-09-21T23:13:05.415035+0000","last_unstale":"2024-09-21T23:13:18.894136+0000","last_undegraded":"2024-09-21T23:13:18.894136+0000","last_fullsized":"2024-09-21T23:13:18.894136+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:58.313765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043940419999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3092,"num_read_kb":25059,"num_write":602,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":379,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893977+0000","last_change":"2024-09-21T23:13:03.643721+0000","last_active":"2024-09-21T23:13:18.893977+0000","last_peered":"2024-09-21T23:13:18.893977+0000","last_clean":"2024-09-21T23:13:18.893977+0000","last_became_active":"2024-09-21T23:13:03.643364+0000","last_became_peered":"2024-09-21T23:13:03.643364+0000","last_unstale":"2024-09-21T23:13:18.893977+0000","last_undegraded":"2024-09-21T23:13:18.893977+0000","last_fullsized":"2024-09-21T23:13:18.893977+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:36.821357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"178'773","reported_seq":5979,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776076+0000","last_change":"2024-09-21T23:12:59.620332+0000","last_active":"2024-09-21T23:13:18.776076+0000","last_peered":"2024-09-21T23:13:18.776076+0000","last_clean":"2024-09-21T23:13:18.776076+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:13:18.776076+0000","last_undegraded":"2024-09-21T23:13:18.776076+0000","last_fullsized":"2024-09-21T23:13:18.776076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:08.127063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086643399999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8038,"num_read_kb":30164,"num_write":1035,"num_write_kb":19449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894851+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:13:18.894851+0000","last_peered":"2024-09-21T23:13:18.894851+0000","last_clean":"2024-09-21T23:13:18.894851+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:13:18.894851+0000","last_undegraded":"2024-09-21T23:13:18.894851+0000","last_fullsized":"2024-09-21T23:13:18.894851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"172'667","reported_seq":4018,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893811+0000","last_change":"2024-09-21T23:12:59.620148+0000","last_active":"2024-09-21T23:13:18.893811+0000","last_peered":"2024-09-21T23:13:18.893811+0000","last_clean":"2024-09-21T23:13:18.893811+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:13:18.893811+0000","last_undegraded":"2024-09-21T23:13:18.893811+0000","last_fullsized":"2024-09-21T23:13:18.893811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:04.503101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043881010000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5200,"num_read_kb":24165,"num_write":600,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894080+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:13:18.894080+0000","last_peered":"2024-09-21T23:13:18.894080+0000","last_clean":"2024-09-21T23:13:18.894080+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:13:18.894080+0000","last_undegraded":"2024-09-21T23:13:18.894080+0000","last_fullsized":"2024-09-21T23:13:18.894080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"173'829","reported_seq":6083,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894254+0000","last_change":"2024-09-21T23:12:59.623402+0000","last_active":"2024-09-21T23:13:18.894254+0000","last_peered":"2024-09-21T23:13:18.894254+0000","last_clean":"2024-09-21T23:13:18.894254+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:13:18.894254+0000","last_undegraded":"2024-09-21T23:13:18.894254+0000","last_fullsized":"2024-09-21T23:13:18.894254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:07:48.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079884020000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6855,"num_read_kb":41392,"num_write":1124,"num_write_kb":18322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'833","reported_seq":4645,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773048+0000","last_change":"2024-09-21T23:12:59.625500+0000","last_active":"2024-09-21T23:13:18.773048+0000","last_peered":"2024-09-21T23:13:18.773048+0000","last_clean":"2024-09-21T23:13:18.773048+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:13:18.773048+0000","last_undegraded":"2024-09-21T23:13:18.773048+0000","last_fullsized":"2024-09-21T23:13:18.773048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:56.312310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014050681000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5872,"num_read_kb":26777,"num_write":811,"num_write_kb":12147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":372,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893953+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:13:18.893953+0000","last_peered":"2024-09-21T23:13:18.893953+0000","last_clean":"2024-09-21T23:13:18.893953+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:13:18.893953+0000","last_undegraded":"2024-09-21T23:13:18.893953+0000","last_fullsized":"2024-09-21T23:13:18.893953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'571","reported_seq":4717,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893469+0000","last_change":"2024-09-21T23:12:59.622124+0000","last_active":"2024-09-21T23:13:18.893469+0000","last_peered":"2024-09-21T23:13:18.893469+0000","last_clean":"2024-09-21T23:13:18.893469+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:13:18.893469+0000","last_undegraded":"2024-09-21T23:13:18.893469+0000","last_fullsized":"2024-09-21T23:13:18.893469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:02.058399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061931130000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6676,"num_read_kb":20540,"num_write":495,"num_write_kb":13141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.774496+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:13:18.774496+0000","last_peered":"2024-09-21T23:13:18.774496+0000","last_clean":"2024-09-21T23:13:18.774496+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:13:18.774496+0000","last_undegraded":"2024-09-21T23:13:18.774496+0000","last_fullsized":"2024-09-21T23:13:18.774496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"177'464","reported_seq":5702,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893865+0000","last_change":"2024-09-21T23:13:05.423253+0000","last_active":"2024-09-21T23:13:18.893865+0000","last_peered":"2024-09-21T23:13:18.893865+0000","last_clean":"2024-09-21T23:13:18.893865+0000","last_became_active":"2024-09-21T23:13:05.421575+0000","last_became_peered":"2024-09-21T23:13:05.421575+0000","last_unstale":"2024-09-21T23:13:18.893865+0000","last_undegraded":"2024-09-21T23:13:18.893865+0000","last_fullsized":"2024-09-21T23:13:18.893865+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:50.035868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013442716,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4762,"num_read_kb":20313,"num_write":497,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"178'583","reported_seq":4474,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775508+0000","last_change":"2024-09-21T23:12:59.618427+0000","last_active":"2024-09-21T23:13:18.775508+0000","last_peered":"2024-09-21T23:13:18.775508+0000","last_clean":"2024-09-21T23:13:18.775508+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:13:18.775508+0000","last_undegraded":"2024-09-21T23:13:18.775508+0000","last_fullsized":"2024-09-21T23:13:18.775508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:08:21.796460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070044759999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5321,"num_read_kb":19543,"num_write":664,"num_write_kb":14858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":371,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893997+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:13:18.893997+0000","last_peered":"2024-09-21T23:13:18.893997+0000","last_clean":"2024-09-21T23:13:18.893997+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:13:18.893997+0000","last_undegraded":"2024-09-21T23:13:18.893997+0000","last_fullsized":"2024-09-21T23:13:18.893997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":470,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893940+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:13:18.893940+0000","last_peered":"2024-09-21T23:13:18.893940+0000","last_clean":"2024-09-21T23:13:18.893940+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:13:18.893940+0000","last_undegraded":"2024-09-21T23:13:18.893940+0000","last_fullsized":"2024-09-21T23:13:18.893940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'543","reported_seq":3852,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775721+0000","last_change":"2024-09-21T23:12:59.615280+0000","last_active":"2024-09-21T23:13:18.775721+0000","last_peered":"2024-09-21T23:13:18.775721+0000","last_clean":"2024-09-21T23:13:18.775721+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:13:18.775721+0000","last_undegraded":"2024-09-21T23:13:18.775721+0000","last_fullsized":"2024-09-21T23:13:18.775721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:55:07.747469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040489300000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4907,"num_read_kb":10236,"num_write":527,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"174'562","reported_seq":4608,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772591+0000","last_change":"2024-09-21T23:12:59.616991+0000","last_active":"2024-09-21T23:13:18.772591+0000","last_peered":"2024-09-21T23:13:18.772591+0000","last_clean":"2024-09-21T23:13:18.772591+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:13:18.772591+0000","last_undegraded":"2024-09-21T23:13:18.772591+0000","last_fullsized":"2024-09-21T23:13:18.772591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:16.424894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062271990000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6420,"num_read_kb":19858,"num_write":506,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"177'650","reported_seq":6686,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895221+0000","last_change":"2024-09-21T23:13:07.427624+0000","last_active":"2024-09-21T23:13:18.895221+0000","last_peered":"2024-09-21T23:13:18.895221+0000","last_clean":"2024-09-21T23:13:18.895221+0000","last_became_active":"2024-09-21T23:13:07.423010+0000","last_became_peered":"2024-09-21T23:13:07.423010+0000","last_unstale":"2024-09-21T23:13:18.895221+0000","last_undegraded":"2024-09-21T23:13:18.895221+0000","last_fullsized":"2024-09-21T23:13:18.895221+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:00.226294+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059613299999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":6967,"num_read_kb":40354,"num_write":560,"num_write_kb":11935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"174'417","reported_seq":3793,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775569+0000","last_change":"2024-09-21T23:12:59.620627+0000","last_active":"2024-09-21T23:13:18.775569+0000","last_peered":"2024-09-21T23:13:18.775569+0000","last_clean":"2024-09-21T23:13:18.775569+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:13:18.775569+0000","last_undegraded":"2024-09-21T23:13:18.775569+0000","last_fullsized":"2024-09-21T23:13:18.775569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:19.532943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009141883,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3542,"num_read_kb":26420,"num_write":356,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'558","reported_seq":2492,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894778+0000","last_change":"2024-09-21T23:12:59.620407+0000","last_active":"2024-09-21T23:13:18.894778+0000","last_peered":"2024-09-21T23:13:18.894778+0000","last_clean":"2024-09-21T23:13:18.894778+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:13:18.894778+0000","last_undegraded":"2024-09-21T23:13:18.894778+0000","last_fullsized":"2024-09-21T23:13:18.894778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:14.289431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051982479999999999,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2495,"num_read_kb":17923,"num_write":507,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'537","reported_seq":5293,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894362+0000","last_change":"2024-09-21T23:12:59.623819+0000","last_active":"2024-09-21T23:13:18.894362+0000","last_peered":"2024-09-21T23:13:18.894362+0000","last_clean":"2024-09-21T23:13:18.894362+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:13:18.894362+0000","last_undegraded":"2024-09-21T23:13:18.894362+0000","last_fullsized":"2024-09-21T23:13:18.894362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:17.779188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004612685,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5087,"num_read_kb":28815,"num_write":438,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"174'511","reported_seq":5210,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894486+0000","last_change":"2024-09-21T23:12:59.622033+0000","last_active":"2024-09-21T23:13:18.894486+0000","last_peered":"2024-09-21T23:13:18.894486+0000","last_clean":"2024-09-21T23:13:18.894486+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:13:18.894486+0000","last_undegraded":"2024-09-21T23:13:18.894486+0000","last_fullsized":"2024-09-21T23:13:18.894486+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:26.432233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033040949999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6927,"num_read_kb":23882,"num_write":481,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"173'593","reported_seq":2676,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894500+0000","last_change":"2024-09-21T23:12:59.621649+0000","last_active":"2024-09-21T23:13:18.894500+0000","last_peered":"2024-09-21T23:13:18.894500+0000","last_clean":"2024-09-21T23:13:18.894500+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:13:18.894500+0000","last_undegraded":"2024-09-21T23:13:18.894500+0000","last_fullsized":"2024-09-21T23:13:18.894500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:34.711481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027323970000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2413,"num_read_kb":21711,"num_write":633,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'671","reported_seq":4520,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895178+0000","last_change":"2024-09-21T23:13:06.426028+0000","last_active":"2024-09-21T23:13:18.895178+0000","last_peered":"2024-09-21T23:13:18.895178+0000","last_clean":"2024-09-21T23:13:18.895178+0000","last_became_active":"2024-09-21T23:13:06.420853+0000","last_became_peered":"2024-09-21T23:13:06.420853+0000","last_unstale":"2024-09-21T23:13:18.895178+0000","last_undegraded":"2024-09-21T23:13:18.895178+0000","last_fullsized":"2024-09-21T23:13:18.895178+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:29.389023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062394340000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6106,"num_read_kb":37597,"num_write":682,"num_write_kb":13471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"172'641","reported_seq":7006,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:12:59.618210+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:43.253945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011826829999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":7995,"num_read_kb":42900,"num_write":666,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'627","reported_seq":7307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893948+0000","last_change":"2024-09-21T23:13:05.416690+0000","last_active":"2024-09-21T23:13:18.893948+0000","last_peered":"2024-09-21T23:13:18.893948+0000","last_clean":"2024-09-21T23:13:18.893948+0000","last_became_active":"2024-09-21T23:13:05.415127+0000","last_became_peered":"2024-09-21T23:13:05.415127+0000","last_unstale":"2024-09-21T23:13:18.893948+0000","last_undegraded":"2024-09-21T23:13:18.893948+0000","last_fullsized":"2024-09-21T23:13:18.893948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:27.939279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032033349999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8199,"num_read_kb":39748,"num_write":659,"num_write_kb":21750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'413","reported_seq":2638,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894555+0000","last_change":"2024-09-21T23:12:59.620277+0000","last_active":"2024-09-21T23:13:18.894555+0000","last_peered":"2024-09-21T23:13:18.894555+0000","last_clean":"2024-09-21T23:13:18.894555+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:13:18.894555+0000","last_undegraded":"2024-09-21T23:13:18.894555+0000","last_fullsized":"2024-09-21T23:13:18.894555+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:07.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562690000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":1457,"num_read_kb":14145,"num_write":378,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":720,"num_read":186161,"num_read_kb":840774,"num_write":22415,"num_write_kb":399960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5301,"num_bytes_recovered":254407838,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":20934,"ondisk_log_size":20934,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29332,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342508,"statfs":{"total":96636764160,"available":96606728192,"internally_reserved":0,"allocated":2355200,"data_stored":1143969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743454,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43424,"kb_used_data":2316,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94328416,"statfs":{"total":96636764160,"available":96592297984,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77916,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293924,"statfs":{"total":96636764160,"available":96556978176,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29340,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342500,"statfs":{"total":96636764160,"available":96606720000,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55716,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316124,"statfs":{"total":96636764160,"available":96579710976,"internally_reserved":0,"allocated":2375680,"data_stored":1149820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:13:22.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:13:22.486 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:22.486 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:13:22.502 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-21T23:13:20.547590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":728,"num_whiteouts":720,"num_read":186277,"num_read_kb":840874,"num_write":22576,"num_write_kb":402378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5305,"num_bytes_recovered":255588574,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21054,"ondisk_log_size":21054,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":390148,"kb_used_data":19660,"kb_used_omap":25,"kb_used_meta":370278,"kb_avail":754584572,"statfs":{"total":773094113280,"available":772694601728,"internally_reserved":0,"allocated":20131840,"data_stored":10373255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":379164719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.807217"},"pg_stats":[{"pgid":"3.1f","version":"178'580","reported_seq":4220,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894261+0000","last_change":"2024-09-21T23:13:05.425494+0000","last_active":"2024-09-21T23:13:18.894261+0000","last_peered":"2024-09-21T23:13:18.894261+0000","last_clean":"2024-09-21T23:13:18.894261+0000","last_became_active":"2024-09-21T23:13:05.423231+0000","last_became_peered":"2024-09-21T23:13:05.423231+0000","last_unstale":"2024-09-21T23:13:18.894261+0000","last_undegraded":"2024-09-21T23:13:18.894261+0000","last_fullsized":"2024-09-21T23:13:18.894261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:12:08.608124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087587719999999997,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6355,"num_read_kb":19114,"num_write":548,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"172'894","reported_seq":4765,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776370+0000","last_change":"2024-09-21T23:13:07.423842+0000","last_active":"2024-09-21T23:13:18.776370+0000","last_peered":"2024-09-21T23:13:18.776370+0000","last_clean":"2024-09-21T23:13:18.776370+0000","last_became_active":"2024-09-21T23:13:07.421837+0000","last_became_peered":"2024-09-21T23:13:07.421837+0000","last_unstale":"2024-09-21T23:13:18.776370+0000","last_undegraded":"2024-09-21T23:13:18.776370+0000","last_fullsized":"2024-09-21T23:13:18.776370+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:26.907455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001882955,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5630,"num_read_kb":41276,"num_write":1148,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'456","reported_seq":3457,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894172+0000","last_change":"2024-09-21T23:12:59.616455+0000","last_active":"2024-09-21T23:13:18.894172+0000","last_peered":"2024-09-21T23:13:18.894172+0000","last_clean":"2024-09-21T23:13:18.894172+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:13:18.894172+0000","last_undegraded":"2024-09-21T23:13:18.894172+0000","last_fullsized":"2024-09-21T23:13:18.894172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:55.115242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158018,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3362,"num_read_kb":13834,"num_write":377,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'500","reported_seq":5839,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894768+0000","last_change":"2024-09-21T23:12:59.618158+0000","last_active":"2024-09-21T23:13:18.894768+0000","last_peered":"2024-09-21T23:13:18.894768+0000","last_clean":"2024-09-21T23:13:18.894768+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:13:18.894768+0000","last_undegraded":"2024-09-21T23:13:18.894768+0000","last_fullsized":"2024-09-21T23:13:18.894768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:16.558115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037616780000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5378,"num_read_kb":22578,"num_write":533,"num_write_kb":12630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"172'618","reported_seq":4487,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894025+0000","last_change":"2024-09-21T23:13:05.423467+0000","last_active":"2024-09-21T23:13:18.894025+0000","last_peered":"2024-09-21T23:13:18.894025+0000","last_clean":"2024-09-21T23:13:18.894025+0000","last_became_active":"2024-09-21T23:13:05.421427+0000","last_became_peered":"2024-09-21T23:13:05.421427+0000","last_unstale":"2024-09-21T23:13:18.894025+0000","last_undegraded":"2024-09-21T23:13:18.894025+0000","last_fullsized":"2024-09-21T23:13:18.894025+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:42.645544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010738146,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5513,"num_read_kb":28043,"num_write":694,"num_write_kb":17938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'1323","reported_seq":7922,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772720+0000","last_change":"2024-09-21T23:12:59.615466+0000","last_active":"2024-09-21T23:13:18.772720+0000","last_peered":"2024-09-21T23:13:18.772720+0000","last_clean":"2024-09-21T23:13:18.772720+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:13:18.772720+0000","last_undegraded":"2024-09-21T23:13:18.772720+0000","last_fullsized":"2024-09-21T23:13:18.772720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:14.383756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046765799999999996,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":12289,"num_read_kb":29931,"num_write":1690,"num_write_kb":9650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"172'1009","reported_seq":3157,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893498+0000","last_change":"2024-09-21T23:12:59.622668+0000","last_active":"2024-09-21T23:13:18.893498+0000","last_peered":"2024-09-21T23:13:18.893498+0000","last_clean":"2024-09-21T23:13:18.893498+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:13:18.893498+0000","last_undegraded":"2024-09-21T23:13:18.893498+0000","last_fullsized":"2024-09-21T23:13:18.893498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:39.668516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006449682,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6326,"num_read_kb":21971,"num_write":1459,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"172'1057","reported_seq":5547,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775421+0000","last_change":"2024-09-21T23:12:59.613494+0000","last_active":"2024-09-21T23:13:18.775421+0000","last_peered":"2024-09-21T23:13:18.775421+0000","last_clean":"2024-09-21T23:13:18.775421+0000","last_became_active":"2024-09-21T23:12:37.444303+0000","last_became_peered":"2024-09-21T23:12:37.444303+0000","last_unstale":"2024-09-21T23:13:18.775421+0000","last_undegraded":"2024-09-21T23:13:18.775421+0000","last_fullsized":"2024-09-21T23:13:18.775421+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:05.804247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030213219999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8120,"num_read_kb":28946,"num_write":1438,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5999502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'652","reported_seq":5692,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894396+0000","last_change":"2024-09-21T23:13:05.425470+0000","last_active":"2024-09-21T23:13:18.894396+0000","last_peered":"2024-09-21T23:13:18.894396+0000","last_clean":"2024-09-21T23:13:18.894396+0000","last_became_active":"2024-09-21T23:13:05.423133+0000","last_became_peered":"2024-09-21T23:13:05.423133+0000","last_unstale":"2024-09-21T23:13:18.894396+0000","last_undegraded":"2024-09-21T23:13:18.894396+0000","last_fullsized":"2024-09-21T23:13:18.894396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:38.996927+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022352600000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":6037,"num_read_kb":23064,"num_write":602,"num_write_kb":12198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'521","reported_seq":4585,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894097+0000","last_change":"2024-09-21T23:12:59.621765+0000","last_active":"2024-09-21T23:13:18.894097+0000","last_peered":"2024-09-21T23:13:18.894097+0000","last_clean":"2024-09-21T23:13:18.894097+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:13:18.894097+0000","last_undegraded":"2024-09-21T23:13:18.894097+0000","last_fullsized":"2024-09-21T23:13:18.894097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:11.726440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066845790000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5732,"num_read_kb":29704,"num_write":420,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"178'561","reported_seq":6978,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894886+0000","last_change":"2024-09-21T23:12:59.624274+0000","last_active":"2024-09-21T23:13:18.894886+0000","last_peered":"2024-09-21T23:13:18.894886+0000","last_clean":"2024-09-21T23:13:18.894886+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:13:18.894886+0000","last_undegraded":"2024-09-21T23:13:18.894886+0000","last_fullsized":"2024-09-21T23:13:18.894886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:35.264209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048388729999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7517,"num_read_kb":36655,"num_write":459,"num_write_kb":15039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'703","reported_seq":5090,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773133+0000","last_change":"2024-09-21T23:12:59.613161+0000","last_active":"2024-09-21T23:13:18.773133+0000","last_peered":"2024-09-21T23:13:18.773133+0000","last_clean":"2024-09-21T23:13:18.773133+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:13:18.773133+0000","last_undegraded":"2024-09-21T23:13:18.773133+0000","last_fullsized":"2024-09-21T23:13:18.773133+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:56.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024929100000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5571,"num_read_kb":14116,"num_write":826,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895381+0000","last_change":"2024-09-21T23:13:03.807826+0000","last_active":"2024-09-21T23:13:18.895381+0000","last_peered":"2024-09-21T23:13:18.895381+0000","last_clean":"2024-09-21T23:13:18.895381+0000","last_became_active":"2024-09-21T23:13:03.806956+0000","last_became_peered":"2024-09-21T23:13:03.806956+0000","last_unstale":"2024-09-21T23:13:18.895381+0000","last_undegraded":"2024-09-21T23:13:18.895381+0000","last_fullsized":"2024-09-21T23:13:18.895381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:13.318790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'617","reported_seq":2796,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894136+0000","last_change":"2024-09-21T23:13:05.416619+0000","last_active":"2024-09-21T23:13:18.894136+0000","last_peered":"2024-09-21T23:13:18.894136+0000","last_clean":"2024-09-21T23:13:18.894136+0000","last_became_active":"2024-09-21T23:13:05.415035+0000","last_became_peered":"2024-09-21T23:13:05.415035+0000","last_unstale":"2024-09-21T23:13:18.894136+0000","last_undegraded":"2024-09-21T23:13:18.894136+0000","last_fullsized":"2024-09-21T23:13:18.894136+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:58.313765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043940419999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3092,"num_read_kb":25059,"num_write":602,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":379,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893977+0000","last_change":"2024-09-21T23:13:03.643721+0000","last_active":"2024-09-21T23:13:18.893977+0000","last_peered":"2024-09-21T23:13:18.893977+0000","last_clean":"2024-09-21T23:13:18.893977+0000","last_became_active":"2024-09-21T23:13:03.643364+0000","last_became_peered":"2024-09-21T23:13:03.643364+0000","last_unstale":"2024-09-21T23:13:18.893977+0000","last_undegraded":"2024-09-21T23:13:18.893977+0000","last_fullsized":"2024-09-21T23:13:18.893977+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:36.821357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"178'773","reported_seq":5979,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776076+0000","last_change":"2024-09-21T23:12:59.620332+0000","last_active":"2024-09-21T23:13:18.776076+0000","last_peered":"2024-09-21T23:13:18.776076+0000","last_clean":"2024-09-21T23:13:18.776076+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:13:18.776076+0000","last_undegraded":"2024-09-21T23:13:18.776076+0000","last_fullsized":"2024-09-21T23:13:18.776076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:08.127063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086643399999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8038,"num_read_kb":30164,"num_write":1035,"num_write_kb":19449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894851+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:13:18.894851+0000","last_peered":"2024-09-21T23:13:18.894851+0000","last_clean":"2024-09-21T23:13:18.894851+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:13:18.894851+0000","last_undegraded":"2024-09-21T23:13:18.894851+0000","last_fullsized":"2024-09-21T23:13:18.894851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"172'667","reported_seq":4018,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893811+0000","last_change":"2024-09-21T23:12:59.620148+0000","last_active":"2024-09-21T23:13:18.893811+0000","last_peered":"2024-09-21T23:13:18.893811+0000","last_clean":"2024-09-21T23:13:18.893811+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:13:18.893811+0000","last_undegraded":"2024-09-21T23:13:18.893811+0000","last_fullsized":"2024-09-21T23:13:18.893811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:04.503101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043881010000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5200,"num_read_kb":24165,"num_write":600,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894080+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:13:18.894080+0000","last_peered":"2024-09-21T23:13:18.894080+0000","last_clean":"2024-09-21T23:13:18.894080+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:13:18.894080+0000","last_undegraded":"2024-09-21T23:13:18.894080+0000","last_fullsized":"2024-09-21T23:13:18.894080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"173'829","reported_seq":6083,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894254+0000","last_change":"2024-09-21T23:12:59.623402+0000","last_active":"2024-09-21T23:13:18.894254+0000","last_peered":"2024-09-21T23:13:18.894254+0000","last_clean":"2024-09-21T23:13:18.894254+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:13:18.894254+0000","last_undegraded":"2024-09-21T23:13:18.894254+0000","last_fullsized":"2024-09-21T23:13:18.894254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:07:48.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079884020000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6855,"num_read_kb":41392,"num_write":1124,"num_write_kb":18322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'833","reported_seq":4645,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773048+0000","last_change":"2024-09-21T23:12:59.625500+0000","last_active":"2024-09-21T23:13:18.773048+0000","last_peered":"2024-09-21T23:13:18.773048+0000","last_clean":"2024-09-21T23:13:18.773048+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:13:18.773048+0000","last_undegraded":"2024-09-21T23:13:18.773048+0000","last_fullsized":"2024-09-21T23:13:18.773048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:56.312310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014050681000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5872,"num_read_kb":26777,"num_write":811,"num_write_kb":12147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":372,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893953+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:13:18.893953+0000","last_peered":"2024-09-21T23:13:18.893953+0000","last_clean":"2024-09-21T23:13:18.893953+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:13:18.893953+0000","last_undegraded":"2024-09-21T23:13:18.893953+0000","last_fullsized":"2024-09-21T23:13:18.893953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'571","reported_seq":4717,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893469+0000","last_change":"2024-09-21T23:12:59.622124+0000","last_active":"2024-09-21T23:13:18.893469+0000","last_peered":"2024-09-21T23:13:18.893469+0000","last_clean":"2024-09-21T23:13:18.893469+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:13:18.893469+0000","last_undegraded":"2024-09-21T23:13:18.893469+0000","last_fullsized":"2024-09-21T23:13:18.893469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:02.058399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061931130000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6676,"num_read_kb":20540,"num_write":495,"num_write_kb":13141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.774496+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:13:18.774496+0000","last_peered":"2024-09-21T23:13:18.774496+0000","last_clean":"2024-09-21T23:13:18.774496+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:13:18.774496+0000","last_undegraded":"2024-09-21T23:13:18.774496+0000","last_fullsized":"2024-09-21T23:13:18.774496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"177'464","reported_seq":5702,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893865+0000","last_change":"2024-09-21T23:13:05.423253+0000","last_active":"2024-09-21T23:13:18.893865+0000","last_peered":"2024-09-21T23:13:18.893865+0000","last_clean":"2024-09-21T23:13:18.893865+0000","last_became_active":"2024-09-21T23:13:05.421575+0000","last_became_peered":"2024-09-21T23:13:05.421575+0000","last_unstale":"2024-09-21T23:13:18.893865+0000","last_undegraded":"2024-09-21T23:13:18.893865+0000","last_fullsized":"2024-09-21T23:13:18.893865+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:50.035868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013442716,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4762,"num_read_kb":20313,"num_write":497,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"178'583","reported_seq":4474,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775508+0000","last_change":"2024-09-21T23:12:59.618427+0000","last_active":"2024-09-21T23:13:18.775508+0000","last_peered":"2024-09-21T23:13:18.775508+0000","last_clean":"2024-09-21T23:13:18.775508+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:13:18.775508+0000","last_undegraded":"2024-09-21T23:13:18.775508+0000","last_fullsized":"2024-09-21T23:13:18.775508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:08:21.796460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070044759999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5321,"num_read_kb":19543,"num_write":664,"num_write_kb":14858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":371,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893997+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:13:18.893997+0000","last_peered":"2024-09-21T23:13:18.893997+0000","last_clean":"2024-09-21T23:13:18.893997+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:13:18.893997+0000","last_undegraded":"2024-09-21T23:13:18.893997+0000","last_fullsized":"2024-09-21T23:13:18.893997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":470,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893940+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:13:18.893940+0000","last_peered":"2024-09-21T23:13:18.893940+0000","last_clean":"2024-09-21T23:13:18.893940+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:13:18.893940+0000","last_undegraded":"2024-09-21T23:13:18.893940+0000","last_fullsized":"2024-09-21T23:13:18.893940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'543","reported_seq":3852,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775721+0000","last_change":"2024-09-21T23:12:59.615280+0000","last_active":"2024-09-21T23:13:18.775721+0000","last_peered":"2024-09-21T23:13:18.775721+0000","last_clean":"2024-09-21T23:13:18.775721+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:13:18.775721+0000","last_undegraded":"2024-09-21T23:13:18.775721+0000","last_fullsized":"2024-09-21T23:13:18.775721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:55:07.747469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040489300000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4907,"num_read_kb":10236,"num_write":527,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"174'562","reported_seq":4608,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772591+0000","last_change":"2024-09-21T23:12:59.616991+0000","last_active":"2024-09-21T23:13:18.772591+0000","last_peered":"2024-09-21T23:13:18.772591+0000","last_clean":"2024-09-21T23:13:18.772591+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:13:18.772591+0000","last_undegraded":"2024-09-21T23:13:18.772591+0000","last_fullsized":"2024-09-21T23:13:18.772591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:16.424894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062271990000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6420,"num_read_kb":19858,"num_write":506,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"177'650","reported_seq":6686,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895221+0000","last_change":"2024-09-21T23:13:07.427624+0000","last_active":"2024-09-21T23:13:18.895221+0000","last_peered":"2024-09-21T23:13:18.895221+0000","last_clean":"2024-09-21T23:13:18.895221+0000","last_became_active":"2024-09-21T23:13:07.423010+0000","last_became_peered":"2024-09-21T23:13:07.423010+0000","last_unstale":"2024-09-21T23:13:18.895221+0000","last_undegraded":"2024-09-21T23:13:18.895221+0000","last_fullsized":"2024-09-21T23:13:18.895221+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:00.226294+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059613299999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":6967,"num_read_kb":40354,"num_write":560,"num_write_kb":11935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"174'417","reported_seq":3793,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775569+0000","last_change":"2024-09-21T23:12:59.620627+0000","last_active":"2024-09-21T23:13:18.775569+0000","last_peered":"2024-09-21T23:13:18.775569+0000","last_clean":"2024-09-21T23:13:18.775569+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:13:18.775569+0000","last_undegraded":"2024-09-21T23:13:18.775569+0000","last_fullsized":"2024-09-21T23:13:18.775569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:19.532943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009141883,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3542,"num_read_kb":26420,"num_write":356,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'558","reported_seq":2492,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894778+0000","last_change":"2024-09-21T23:12:59.620407+0000","last_active":"2024-09-21T23:13:18.894778+0000","last_peered":"2024-09-21T23:13:18.894778+0000","last_clean":"2024-09-21T23:13:18.894778+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:13:18.894778+0000","last_undegraded":"2024-09-21T23:13:18.894778+0000","last_fullsized":"2024-09-21T23:13:18.894778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:14.289431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051982479999999999,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2495,"num_read_kb":17923,"num_write":507,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'537","reported_seq":5293,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894362+0000","last_change":"2024-09-21T23:12:59.623819+0000","last_active":"2024-09-21T23:13:18.894362+0000","last_peered":"2024-09-21T23:13:18.894362+0000","last_clean":"2024-09-21T23:13:18.894362+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:13:18.894362+0000","last_undegraded":"2024-09-21T23:13:18.894362+0000","last_fullsized":"2024-09-21T23:13:18.894362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:17.779188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004612685,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5087,"num_read_kb":28815,"num_write":438,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"174'511","reported_seq":5210,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894486+0000","last_change":"2024-09-21T23:12:59.622033+0000","last_active":"2024-09-21T23:13:18.894486+0000","last_peered":"2024-09-21T23:13:18.894486+0000","last_clean":"2024-09-21T23:13:18.894486+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:13:18.894486+0000","last_undegraded":"2024-09-21T23:13:18.894486+0000","last_fullsized":"2024-09-21T23:13:18.894486+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:26.432233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033040949999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6927,"num_read_kb":23882,"num_write":481,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"173'593","reported_seq":2676,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894500+0000","last_change":"2024-09-21T23:12:59.621649+0000","last_active":"2024-09-21T23:13:18.894500+0000","last_peered":"2024-09-21T23:13:18.894500+0000","last_clean":"2024-09-21T23:13:18.894500+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:13:18.894500+0000","last_undegraded":"2024-09-21T23:13:18.894500+0000","last_fullsized":"2024-09-21T23:13:18.894500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:34.711481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027323970000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2413,"num_read_kb":21711,"num_write":633,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'671","reported_seq":4520,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895178+0000","last_change":"2024-09-21T23:13:06.426028+0000","last_active":"2024-09-21T23:13:18.895178+0000","last_peered":"2024-09-21T23:13:18.895178+0000","last_clean":"2024-09-21T23:13:18.895178+0000","last_became_active":"2024-09-21T23:13:06.420853+0000","last_became_peered":"2024-09-21T23:13:06.420853+0000","last_unstale":"2024-09-21T23:13:18.895178+0000","last_undegraded":"2024-09-21T23:13:18.895178+0000","last_fullsized":"2024-09-21T23:13:18.895178+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:29.389023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062394340000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6106,"num_read_kb":37597,"num_write":682,"num_write_kb":13471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"172'641","reported_seq":7006,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:12:59.618210+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:43.253945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011826829999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":7995,"num_read_kb":42900,"num_write":666,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'627","reported_seq":7307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893948+0000","last_change":"2024-09-21T23:13:05.416690+0000","last_active":"2024-09-21T23:13:18.893948+0000","last_peered":"2024-09-21T23:13:18.893948+0000","last_clean":"2024-09-21T23:13:18.893948+0000","last_became_active":"2024-09-21T23:13:05.415127+0000","last_became_peered":"2024-09-21T23:13:05.415127+0000","last_unstale":"2024-09-21T23:13:18.893948+0000","last_undegraded":"2024-09-21T23:13:18.893948+0000","last_fullsized":"2024-09-21T23:13:18.893948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:27.939279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032033349999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8199,"num_read_kb":39748,"num_write":659,"num_write_kb":21750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'413","reported_seq":2638,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894555+0000","last_change":"2024-09-21T23:12:59.620277+0000","last_active":"2024-09-21T23:13:18.894555+0000","last_peered":"2024-09-21T23:13:18.894555+0000","last_clean":"2024-09-21T23:13:18.894555+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:13:18.894555+0000","last_undegraded":"2024-09-21T23:13:18.894555+0000","last_fullsized":"2024-09-21T23:13:18.894555+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:07.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562690000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":1457,"num_read_kb":14145,"num_write":378,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":720,"num_read":186161,"num_read_kb":840774,"num_write":22415,"num_write_kb":399960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5301,"num_bytes_recovered":254407838,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":20934,"ondisk_log_size":20934,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29332,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342508,"statfs":{"total":96636764160,"available":96606728192,"internally_reserved":0,"allocated":2355200,"data_stored":1143969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743454,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43424,"kb_used_data":2316,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94328416,"statfs":{"total":96636764160,"available":96592297984,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77916,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293924,"statfs":{"total":96636764160,"available":96556978176,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29340,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342500,"statfs":{"total":96636764160,"available":96606720000,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55716,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316124,"statfs":{"total":96636764160,"available":96579710976,"internally_reserved":0,"allocated":2375680,"data_stored":1149820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:13:22.505 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:13:22.505 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T23:13:22.804 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T23:13:22.815 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:13:22.816 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:13:23.103 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:23.104 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:13:23.121 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-21T23:13:22.548291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":728,"num_whiteouts":720,"num_read":186277,"num_read_kb":840874,"num_write":22576,"num_write_kb":402378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5305,"num_bytes_recovered":255588574,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21054,"ondisk_log_size":21054,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":390164,"kb_used_data":19672,"kb_used_omap":25,"kb_used_meta":370278,"kb_avail":754584556,"statfs":{"total":773094113280,"available":772694585344,"internally_reserved":0,"allocated":20144128,"data_stored":10379068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":379164719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001805"},"pg_stats":[{"pgid":"3.1f","version":"178'580","reported_seq":4220,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894261+0000","last_change":"2024-09-21T23:13:05.425494+0000","last_active":"2024-09-21T23:13:18.894261+0000","last_peered":"2024-09-21T23:13:18.894261+0000","last_clean":"2024-09-21T23:13:18.894261+0000","last_became_active":"2024-09-21T23:13:05.423231+0000","last_became_peered":"2024-09-21T23:13:05.423231+0000","last_unstale":"2024-09-21T23:13:18.894261+0000","last_undegraded":"2024-09-21T23:13:18.894261+0000","last_fullsized":"2024-09-21T23:13:18.894261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:12:08.608124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087587719999999997,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6355,"num_read_kb":19114,"num_write":548,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"172'894","reported_seq":4765,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776370+0000","last_change":"2024-09-21T23:13:07.423842+0000","last_active":"2024-09-21T23:13:18.776370+0000","last_peered":"2024-09-21T23:13:18.776370+0000","last_clean":"2024-09-21T23:13:18.776370+0000","last_became_active":"2024-09-21T23:13:07.421837+0000","last_became_peered":"2024-09-21T23:13:07.421837+0000","last_unstale":"2024-09-21T23:13:18.776370+0000","last_undegraded":"2024-09-21T23:13:18.776370+0000","last_fullsized":"2024-09-21T23:13:18.776370+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:26.907455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001882955,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5630,"num_read_kb":41276,"num_write":1148,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'456","reported_seq":3457,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894172+0000","last_change":"2024-09-21T23:12:59.616455+0000","last_active":"2024-09-21T23:13:18.894172+0000","last_peered":"2024-09-21T23:13:18.894172+0000","last_clean":"2024-09-21T23:13:18.894172+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:13:18.894172+0000","last_undegraded":"2024-09-21T23:13:18.894172+0000","last_fullsized":"2024-09-21T23:13:18.894172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:55.115242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158018,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3362,"num_read_kb":13834,"num_write":377,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'500","reported_seq":5839,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894768+0000","last_change":"2024-09-21T23:12:59.618158+0000","last_active":"2024-09-21T23:13:18.894768+0000","last_peered":"2024-09-21T23:13:18.894768+0000","last_clean":"2024-09-21T23:13:18.894768+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:13:18.894768+0000","last_undegraded":"2024-09-21T23:13:18.894768+0000","last_fullsized":"2024-09-21T23:13:18.894768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:16.558115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037616780000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5378,"num_read_kb":22578,"num_write":533,"num_write_kb":12630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"172'618","reported_seq":4487,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894025+0000","last_change":"2024-09-21T23:13:05.423467+0000","last_active":"2024-09-21T23:13:18.894025+0000","last_peered":"2024-09-21T23:13:18.894025+0000","last_clean":"2024-09-21T23:13:18.894025+0000","last_became_active":"2024-09-21T23:13:05.421427+0000","last_became_peered":"2024-09-21T23:13:05.421427+0000","last_unstale":"2024-09-21T23:13:18.894025+0000","last_undegraded":"2024-09-21T23:13:18.894025+0000","last_fullsized":"2024-09-21T23:13:18.894025+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:42.645544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010738146,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5513,"num_read_kb":28043,"num_write":694,"num_write_kb":17938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'1323","reported_seq":7922,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772720+0000","last_change":"2024-09-21T23:12:59.615466+0000","last_active":"2024-09-21T23:13:18.772720+0000","last_peered":"2024-09-21T23:13:18.772720+0000","last_clean":"2024-09-21T23:13:18.772720+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:13:18.772720+0000","last_undegraded":"2024-09-21T23:13:18.772720+0000","last_fullsized":"2024-09-21T23:13:18.772720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:14.383756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046765799999999996,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":12289,"num_read_kb":29931,"num_write":1690,"num_write_kb":9650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"172'1009","reported_seq":3157,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893498+0000","last_change":"2024-09-21T23:12:59.622668+0000","last_active":"2024-09-21T23:13:18.893498+0000","last_peered":"2024-09-21T23:13:18.893498+0000","last_clean":"2024-09-21T23:13:18.893498+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:13:18.893498+0000","last_undegraded":"2024-09-21T23:13:18.893498+0000","last_fullsized":"2024-09-21T23:13:18.893498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:39.668516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006449682,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6326,"num_read_kb":21971,"num_write":1459,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"172'1057","reported_seq":5547,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775421+0000","last_change":"2024-09-21T23:12:59.613494+0000","last_active":"2024-09-21T23:13:18.775421+0000","last_peered":"2024-09-21T23:13:18.775421+0000","last_clean":"2024-09-21T23:13:18.775421+0000","last_became_active":"2024-09-21T23:12:37.444303+0000","last_became_peered":"2024-09-21T23:12:37.444303+0000","last_unstale":"2024-09-21T23:13:18.775421+0000","last_undegraded":"2024-09-21T23:13:18.775421+0000","last_fullsized":"2024-09-21T23:13:18.775421+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:05.804247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030213219999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8120,"num_read_kb":28946,"num_write":1438,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5999502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'652","reported_seq":5692,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894396+0000","last_change":"2024-09-21T23:13:05.425470+0000","last_active":"2024-09-21T23:13:18.894396+0000","last_peered":"2024-09-21T23:13:18.894396+0000","last_clean":"2024-09-21T23:13:18.894396+0000","last_became_active":"2024-09-21T23:13:05.423133+0000","last_became_peered":"2024-09-21T23:13:05.423133+0000","last_unstale":"2024-09-21T23:13:18.894396+0000","last_undegraded":"2024-09-21T23:13:18.894396+0000","last_fullsized":"2024-09-21T23:13:18.894396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:38.996927+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022352600000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":6037,"num_read_kb":23064,"num_write":602,"num_write_kb":12198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'521","reported_seq":4585,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894097+0000","last_change":"2024-09-21T23:12:59.621765+0000","last_active":"2024-09-21T23:13:18.894097+0000","last_peered":"2024-09-21T23:13:18.894097+0000","last_clean":"2024-09-21T23:13:18.894097+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:13:18.894097+0000","last_undegraded":"2024-09-21T23:13:18.894097+0000","last_fullsized":"2024-09-21T23:13:18.894097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:11.726440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066845790000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5732,"num_read_kb":29704,"num_write":420,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"178'561","reported_seq":6978,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894886+0000","last_change":"2024-09-21T23:12:59.624274+0000","last_active":"2024-09-21T23:13:18.894886+0000","last_peered":"2024-09-21T23:13:18.894886+0000","last_clean":"2024-09-21T23:13:18.894886+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:13:18.894886+0000","last_undegraded":"2024-09-21T23:13:18.894886+0000","last_fullsized":"2024-09-21T23:13:18.894886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:35.264209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048388729999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7517,"num_read_kb":36655,"num_write":459,"num_write_kb":15039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'703","reported_seq":5090,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773133+0000","last_change":"2024-09-21T23:12:59.613161+0000","last_active":"2024-09-21T23:13:18.773133+0000","last_peered":"2024-09-21T23:13:18.773133+0000","last_clean":"2024-09-21T23:13:18.773133+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:13:18.773133+0000","last_undegraded":"2024-09-21T23:13:18.773133+0000","last_fullsized":"2024-09-21T23:13:18.773133+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:56.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024929100000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5571,"num_read_kb":14116,"num_write":826,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895381+0000","last_change":"2024-09-21T23:13:03.807826+0000","last_active":"2024-09-21T23:13:18.895381+0000","last_peered":"2024-09-21T23:13:18.895381+0000","last_clean":"2024-09-21T23:13:18.895381+0000","last_became_active":"2024-09-21T23:13:03.806956+0000","last_became_peered":"2024-09-21T23:13:03.806956+0000","last_unstale":"2024-09-21T23:13:18.895381+0000","last_undegraded":"2024-09-21T23:13:18.895381+0000","last_fullsized":"2024-09-21T23:13:18.895381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:13.318790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'617","reported_seq":2796,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894136+0000","last_change":"2024-09-21T23:13:05.416619+0000","last_active":"2024-09-21T23:13:18.894136+0000","last_peered":"2024-09-21T23:13:18.894136+0000","last_clean":"2024-09-21T23:13:18.894136+0000","last_became_active":"2024-09-21T23:13:05.415035+0000","last_became_peered":"2024-09-21T23:13:05.415035+0000","last_unstale":"2024-09-21T23:13:18.894136+0000","last_undegraded":"2024-09-21T23:13:18.894136+0000","last_fullsized":"2024-09-21T23:13:18.894136+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:58.313765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043940419999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3092,"num_read_kb":25059,"num_write":602,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":379,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893977+0000","last_change":"2024-09-21T23:13:03.643721+0000","last_active":"2024-09-21T23:13:18.893977+0000","last_peered":"2024-09-21T23:13:18.893977+0000","last_clean":"2024-09-21T23:13:18.893977+0000","last_became_active":"2024-09-21T23:13:03.643364+0000","last_became_peered":"2024-09-21T23:13:03.643364+0000","last_unstale":"2024-09-21T23:13:18.893977+0000","last_undegraded":"2024-09-21T23:13:18.893977+0000","last_fullsized":"2024-09-21T23:13:18.893977+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:36.821357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"178'773","reported_seq":5979,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776076+0000","last_change":"2024-09-21T23:12:59.620332+0000","last_active":"2024-09-21T23:13:18.776076+0000","last_peered":"2024-09-21T23:13:18.776076+0000","last_clean":"2024-09-21T23:13:18.776076+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:13:18.776076+0000","last_undegraded":"2024-09-21T23:13:18.776076+0000","last_fullsized":"2024-09-21T23:13:18.776076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:08.127063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086643399999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8038,"num_read_kb":30164,"num_write":1035,"num_write_kb":19449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894851+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:13:18.894851+0000","last_peered":"2024-09-21T23:13:18.894851+0000","last_clean":"2024-09-21T23:13:18.894851+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:13:18.894851+0000","last_undegraded":"2024-09-21T23:13:18.894851+0000","last_fullsized":"2024-09-21T23:13:18.894851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"172'667","reported_seq":4018,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893811+0000","last_change":"2024-09-21T23:12:59.620148+0000","last_active":"2024-09-21T23:13:18.893811+0000","last_peered":"2024-09-21T23:13:18.893811+0000","last_clean":"2024-09-21T23:13:18.893811+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:13:18.893811+0000","last_undegraded":"2024-09-21T23:13:18.893811+0000","last_fullsized":"2024-09-21T23:13:18.893811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:04.503101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043881010000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5200,"num_read_kb":24165,"num_write":600,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894080+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:13:18.894080+0000","last_peered":"2024-09-21T23:13:18.894080+0000","last_clean":"2024-09-21T23:13:18.894080+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:13:18.894080+0000","last_undegraded":"2024-09-21T23:13:18.894080+0000","last_fullsized":"2024-09-21T23:13:18.894080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"173'829","reported_seq":6083,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894254+0000","last_change":"2024-09-21T23:12:59.623402+0000","last_active":"2024-09-21T23:13:18.894254+0000","last_peered":"2024-09-21T23:13:18.894254+0000","last_clean":"2024-09-21T23:13:18.894254+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:13:18.894254+0000","last_undegraded":"2024-09-21T23:13:18.894254+0000","last_fullsized":"2024-09-21T23:13:18.894254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:07:48.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079884020000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6855,"num_read_kb":41392,"num_write":1124,"num_write_kb":18322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'833","reported_seq":4645,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773048+0000","last_change":"2024-09-21T23:12:59.625500+0000","last_active":"2024-09-21T23:13:18.773048+0000","last_peered":"2024-09-21T23:13:18.773048+0000","last_clean":"2024-09-21T23:13:18.773048+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:13:18.773048+0000","last_undegraded":"2024-09-21T23:13:18.773048+0000","last_fullsized":"2024-09-21T23:13:18.773048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:56.312310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014050681000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5872,"num_read_kb":26777,"num_write":811,"num_write_kb":12147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":372,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893953+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:13:18.893953+0000","last_peered":"2024-09-21T23:13:18.893953+0000","last_clean":"2024-09-21T23:13:18.893953+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:13:18.893953+0000","last_undegraded":"2024-09-21T23:13:18.893953+0000","last_fullsized":"2024-09-21T23:13:18.893953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'571","reported_seq":4717,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893469+0000","last_change":"2024-09-21T23:12:59.622124+0000","last_active":"2024-09-21T23:13:18.893469+0000","last_peered":"2024-09-21T23:13:18.893469+0000","last_clean":"2024-09-21T23:13:18.893469+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:13:18.893469+0000","last_undegraded":"2024-09-21T23:13:18.893469+0000","last_fullsized":"2024-09-21T23:13:18.893469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:02.058399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061931130000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6676,"num_read_kb":20540,"num_write":495,"num_write_kb":13141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.774496+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:13:18.774496+0000","last_peered":"2024-09-21T23:13:18.774496+0000","last_clean":"2024-09-21T23:13:18.774496+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:13:18.774496+0000","last_undegraded":"2024-09-21T23:13:18.774496+0000","last_fullsized":"2024-09-21T23:13:18.774496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"177'464","reported_seq":5702,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893865+0000","last_change":"2024-09-21T23:13:05.423253+0000","last_active":"2024-09-21T23:13:18.893865+0000","last_peered":"2024-09-21T23:13:18.893865+0000","last_clean":"2024-09-21T23:13:18.893865+0000","last_became_active":"2024-09-21T23:13:05.421575+0000","last_became_peered":"2024-09-21T23:13:05.421575+0000","last_unstale":"2024-09-21T23:13:18.893865+0000","last_undegraded":"2024-09-21T23:13:18.893865+0000","last_fullsized":"2024-09-21T23:13:18.893865+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:50.035868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013442716,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4762,"num_read_kb":20313,"num_write":497,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"178'583","reported_seq":4474,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775508+0000","last_change":"2024-09-21T23:12:59.618427+0000","last_active":"2024-09-21T23:13:18.775508+0000","last_peered":"2024-09-21T23:13:18.775508+0000","last_clean":"2024-09-21T23:13:18.775508+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:13:18.775508+0000","last_undegraded":"2024-09-21T23:13:18.775508+0000","last_fullsized":"2024-09-21T23:13:18.775508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:08:21.796460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070044759999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5321,"num_read_kb":19543,"num_write":664,"num_write_kb":14858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":371,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893997+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:13:18.893997+0000","last_peered":"2024-09-21T23:13:18.893997+0000","last_clean":"2024-09-21T23:13:18.893997+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:13:18.893997+0000","last_undegraded":"2024-09-21T23:13:18.893997+0000","last_fullsized":"2024-09-21T23:13:18.893997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":470,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893940+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:13:18.893940+0000","last_peered":"2024-09-21T23:13:18.893940+0000","last_clean":"2024-09-21T23:13:18.893940+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:13:18.893940+0000","last_undegraded":"2024-09-21T23:13:18.893940+0000","last_fullsized":"2024-09-21T23:13:18.893940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'543","reported_seq":3852,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775721+0000","last_change":"2024-09-21T23:12:59.615280+0000","last_active":"2024-09-21T23:13:18.775721+0000","last_peered":"2024-09-21T23:13:18.775721+0000","last_clean":"2024-09-21T23:13:18.775721+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:13:18.775721+0000","last_undegraded":"2024-09-21T23:13:18.775721+0000","last_fullsized":"2024-09-21T23:13:18.775721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:55:07.747469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040489300000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4907,"num_read_kb":10236,"num_write":527,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"174'562","reported_seq":4608,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772591+0000","last_change":"2024-09-21T23:12:59.616991+0000","last_active":"2024-09-21T23:13:18.772591+0000","last_peered":"2024-09-21T23:13:18.772591+0000","last_clean":"2024-09-21T23:13:18.772591+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:13:18.772591+0000","last_undegraded":"2024-09-21T23:13:18.772591+0000","last_fullsized":"2024-09-21T23:13:18.772591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:16.424894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062271990000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6420,"num_read_kb":19858,"num_write":506,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"177'650","reported_seq":6686,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895221+0000","last_change":"2024-09-21T23:13:07.427624+0000","last_active":"2024-09-21T23:13:18.895221+0000","last_peered":"2024-09-21T23:13:18.895221+0000","last_clean":"2024-09-21T23:13:18.895221+0000","last_became_active":"2024-09-21T23:13:07.423010+0000","last_became_peered":"2024-09-21T23:13:07.423010+0000","last_unstale":"2024-09-21T23:13:18.895221+0000","last_undegraded":"2024-09-21T23:13:18.895221+0000","last_fullsized":"2024-09-21T23:13:18.895221+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:00.226294+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059613299999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":6967,"num_read_kb":40354,"num_write":560,"num_write_kb":11935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"174'417","reported_seq":3793,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775569+0000","last_change":"2024-09-21T23:12:59.620627+0000","last_active":"2024-09-21T23:13:18.775569+0000","last_peered":"2024-09-21T23:13:18.775569+0000","last_clean":"2024-09-21T23:13:18.775569+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:13:18.775569+0000","last_undegraded":"2024-09-21T23:13:18.775569+0000","last_fullsized":"2024-09-21T23:13:18.775569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:19.532943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009141883,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3542,"num_read_kb":26420,"num_write":356,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'558","reported_seq":2492,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894778+0000","last_change":"2024-09-21T23:12:59.620407+0000","last_active":"2024-09-21T23:13:18.894778+0000","last_peered":"2024-09-21T23:13:18.894778+0000","last_clean":"2024-09-21T23:13:18.894778+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:13:18.894778+0000","last_undegraded":"2024-09-21T23:13:18.894778+0000","last_fullsized":"2024-09-21T23:13:18.894778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:14.289431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051982479999999999,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2495,"num_read_kb":17923,"num_write":507,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'537","reported_seq":5293,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894362+0000","last_change":"2024-09-21T23:12:59.623819+0000","last_active":"2024-09-21T23:13:18.894362+0000","last_peered":"2024-09-21T23:13:18.894362+0000","last_clean":"2024-09-21T23:13:18.894362+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:13:18.894362+0000","last_undegraded":"2024-09-21T23:13:18.894362+0000","last_fullsized":"2024-09-21T23:13:18.894362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:17.779188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004612685,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5087,"num_read_kb":28815,"num_write":438,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"174'511","reported_seq":5210,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894486+0000","last_change":"2024-09-21T23:12:59.622033+0000","last_active":"2024-09-21T23:13:18.894486+0000","last_peered":"2024-09-21T23:13:18.894486+0000","last_clean":"2024-09-21T23:13:18.894486+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:13:18.894486+0000","last_undegraded":"2024-09-21T23:13:18.894486+0000","last_fullsized":"2024-09-21T23:13:18.894486+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:26.432233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033040949999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6927,"num_read_kb":23882,"num_write":481,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"173'593","reported_seq":2676,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894500+0000","last_change":"2024-09-21T23:12:59.621649+0000","last_active":"2024-09-21T23:13:18.894500+0000","last_peered":"2024-09-21T23:13:18.894500+0000","last_clean":"2024-09-21T23:13:18.894500+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:13:18.894500+0000","last_undegraded":"2024-09-21T23:13:18.894500+0000","last_fullsized":"2024-09-21T23:13:18.894500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:34.711481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027323970000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2413,"num_read_kb":21711,"num_write":633,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'671","reported_seq":4520,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895178+0000","last_change":"2024-09-21T23:13:06.426028+0000","last_active":"2024-09-21T23:13:18.895178+0000","last_peered":"2024-09-21T23:13:18.895178+0000","last_clean":"2024-09-21T23:13:18.895178+0000","last_became_active":"2024-09-21T23:13:06.420853+0000","last_became_peered":"2024-09-21T23:13:06.420853+0000","last_unstale":"2024-09-21T23:13:18.895178+0000","last_undegraded":"2024-09-21T23:13:18.895178+0000","last_fullsized":"2024-09-21T23:13:18.895178+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:29.389023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062394340000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6106,"num_read_kb":37597,"num_write":682,"num_write_kb":13471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"172'641","reported_seq":7006,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:12:59.618210+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:43.253945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011826829999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":7995,"num_read_kb":42900,"num_write":666,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'627","reported_seq":7307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893948+0000","last_change":"2024-09-21T23:13:05.416690+0000","last_active":"2024-09-21T23:13:18.893948+0000","last_peered":"2024-09-21T23:13:18.893948+0000","last_clean":"2024-09-21T23:13:18.893948+0000","last_became_active":"2024-09-21T23:13:05.415127+0000","last_became_peered":"2024-09-21T23:13:05.415127+0000","last_unstale":"2024-09-21T23:13:18.893948+0000","last_undegraded":"2024-09-21T23:13:18.893948+0000","last_fullsized":"2024-09-21T23:13:18.893948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:27.939279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032033349999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8199,"num_read_kb":39748,"num_write":659,"num_write_kb":21750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'413","reported_seq":2638,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894555+0000","last_change":"2024-09-21T23:12:59.620277+0000","last_active":"2024-09-21T23:13:18.894555+0000","last_peered":"2024-09-21T23:13:18.894555+0000","last_clean":"2024-09-21T23:13:18.894555+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:13:18.894555+0000","last_undegraded":"2024-09-21T23:13:18.894555+0000","last_fullsized":"2024-09-21T23:13:18.894555+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:07.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562690000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":1457,"num_read_kb":14145,"num_write":378,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":720,"num_read":186161,"num_read_kb":840774,"num_write":22415,"num_write_kb":399960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5301,"num_bytes_recovered":254407838,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":20934,"ondisk_log_size":20934,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29348,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342492,"statfs":{"total":96636764160,"available":96606711808,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743454,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43424,"kb_used_data":2316,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94328416,"statfs":{"total":96636764160,"available":96592297984,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77916,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293924,"statfs":{"total":96636764160,"available":96556978176,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29340,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342500,"statfs":{"total":96636764160,"available":96606720000,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55716,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316124,"statfs":{"total":96636764160,"available":96579710976,"internally_reserved":0,"allocated":2375680,"data_stored":1149820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:13:23.123 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:13:23.409 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:23.410 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:13:23.426 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-21T23:13:22.548291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":728,"num_whiteouts":720,"num_read":186277,"num_read_kb":840874,"num_write":22576,"num_write_kb":402378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5305,"num_bytes_recovered":255588574,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21054,"ondisk_log_size":21054,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":390164,"kb_used_data":19672,"kb_used_omap":25,"kb_used_meta":370278,"kb_avail":754584556,"statfs":{"total":773094113280,"available":772694585344,"internally_reserved":0,"allocated":20144128,"data_stored":10379068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":379164719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001805"},"pg_stats":[{"pgid":"3.1f","version":"178'580","reported_seq":4220,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894261+0000","last_change":"2024-09-21T23:13:05.425494+0000","last_active":"2024-09-21T23:13:18.894261+0000","last_peered":"2024-09-21T23:13:18.894261+0000","last_clean":"2024-09-21T23:13:18.894261+0000","last_became_active":"2024-09-21T23:13:05.423231+0000","last_became_peered":"2024-09-21T23:13:05.423231+0000","last_unstale":"2024-09-21T23:13:18.894261+0000","last_undegraded":"2024-09-21T23:13:18.894261+0000","last_fullsized":"2024-09-21T23:13:18.894261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:12:08.608124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087587719999999997,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6355,"num_read_kb":19114,"num_write":548,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"172'894","reported_seq":4765,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776370+0000","last_change":"2024-09-21T23:13:07.423842+0000","last_active":"2024-09-21T23:13:18.776370+0000","last_peered":"2024-09-21T23:13:18.776370+0000","last_clean":"2024-09-21T23:13:18.776370+0000","last_became_active":"2024-09-21T23:13:07.421837+0000","last_became_peered":"2024-09-21T23:13:07.421837+0000","last_unstale":"2024-09-21T23:13:18.776370+0000","last_undegraded":"2024-09-21T23:13:18.776370+0000","last_fullsized":"2024-09-21T23:13:18.776370+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:26.907455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001882955,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5630,"num_read_kb":41276,"num_write":1148,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'456","reported_seq":3457,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894172+0000","last_change":"2024-09-21T23:12:59.616455+0000","last_active":"2024-09-21T23:13:18.894172+0000","last_peered":"2024-09-21T23:13:18.894172+0000","last_clean":"2024-09-21T23:13:18.894172+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:13:18.894172+0000","last_undegraded":"2024-09-21T23:13:18.894172+0000","last_fullsized":"2024-09-21T23:13:18.894172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:55.115242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158018,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3362,"num_read_kb":13834,"num_write":377,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'500","reported_seq":5839,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894768+0000","last_change":"2024-09-21T23:12:59.618158+0000","last_active":"2024-09-21T23:13:18.894768+0000","last_peered":"2024-09-21T23:13:18.894768+0000","last_clean":"2024-09-21T23:13:18.894768+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:13:18.894768+0000","last_undegraded":"2024-09-21T23:13:18.894768+0000","last_fullsized":"2024-09-21T23:13:18.894768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:16.558115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037616780000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5378,"num_read_kb":22578,"num_write":533,"num_write_kb":12630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"172'618","reported_seq":4487,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894025+0000","last_change":"2024-09-21T23:13:05.423467+0000","last_active":"2024-09-21T23:13:18.894025+0000","last_peered":"2024-09-21T23:13:18.894025+0000","last_clean":"2024-09-21T23:13:18.894025+0000","last_became_active":"2024-09-21T23:13:05.421427+0000","last_became_peered":"2024-09-21T23:13:05.421427+0000","last_unstale":"2024-09-21T23:13:18.894025+0000","last_undegraded":"2024-09-21T23:13:18.894025+0000","last_fullsized":"2024-09-21T23:13:18.894025+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:42.645544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010738146,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5513,"num_read_kb":28043,"num_write":694,"num_write_kb":17938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'1323","reported_seq":7922,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772720+0000","last_change":"2024-09-21T23:12:59.615466+0000","last_active":"2024-09-21T23:13:18.772720+0000","last_peered":"2024-09-21T23:13:18.772720+0000","last_clean":"2024-09-21T23:13:18.772720+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:13:18.772720+0000","last_undegraded":"2024-09-21T23:13:18.772720+0000","last_fullsized":"2024-09-21T23:13:18.772720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:14.383756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046765799999999996,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":12289,"num_read_kb":29931,"num_write":1690,"num_write_kb":9650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"172'1009","reported_seq":3157,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893498+0000","last_change":"2024-09-21T23:12:59.622668+0000","last_active":"2024-09-21T23:13:18.893498+0000","last_peered":"2024-09-21T23:13:18.893498+0000","last_clean":"2024-09-21T23:13:18.893498+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:13:18.893498+0000","last_undegraded":"2024-09-21T23:13:18.893498+0000","last_fullsized":"2024-09-21T23:13:18.893498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:39.668516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006449682,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6326,"num_read_kb":21971,"num_write":1459,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"172'1057","reported_seq":5547,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775421+0000","last_change":"2024-09-21T23:12:59.613494+0000","last_active":"2024-09-21T23:13:18.775421+0000","last_peered":"2024-09-21T23:13:18.775421+0000","last_clean":"2024-09-21T23:13:18.775421+0000","last_became_active":"2024-09-21T23:12:37.444303+0000","last_became_peered":"2024-09-21T23:12:37.444303+0000","last_unstale":"2024-09-21T23:13:18.775421+0000","last_undegraded":"2024-09-21T23:13:18.775421+0000","last_fullsized":"2024-09-21T23:13:18.775421+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:05.804247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030213219999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8120,"num_read_kb":28946,"num_write":1438,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5999502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'652","reported_seq":5692,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894396+0000","last_change":"2024-09-21T23:13:05.425470+0000","last_active":"2024-09-21T23:13:18.894396+0000","last_peered":"2024-09-21T23:13:18.894396+0000","last_clean":"2024-09-21T23:13:18.894396+0000","last_became_active":"2024-09-21T23:13:05.423133+0000","last_became_peered":"2024-09-21T23:13:05.423133+0000","last_unstale":"2024-09-21T23:13:18.894396+0000","last_undegraded":"2024-09-21T23:13:18.894396+0000","last_fullsized":"2024-09-21T23:13:18.894396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:38.996927+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022352600000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":6037,"num_read_kb":23064,"num_write":602,"num_write_kb":12198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'521","reported_seq":4585,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894097+0000","last_change":"2024-09-21T23:12:59.621765+0000","last_active":"2024-09-21T23:13:18.894097+0000","last_peered":"2024-09-21T23:13:18.894097+0000","last_clean":"2024-09-21T23:13:18.894097+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:13:18.894097+0000","last_undegraded":"2024-09-21T23:13:18.894097+0000","last_fullsized":"2024-09-21T23:13:18.894097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:11.726440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066845790000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5732,"num_read_kb":29704,"num_write":420,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"178'561","reported_seq":6978,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894886+0000","last_change":"2024-09-21T23:12:59.624274+0000","last_active":"2024-09-21T23:13:18.894886+0000","last_peered":"2024-09-21T23:13:18.894886+0000","last_clean":"2024-09-21T23:13:18.894886+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:13:18.894886+0000","last_undegraded":"2024-09-21T23:13:18.894886+0000","last_fullsized":"2024-09-21T23:13:18.894886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:35.264209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048388729999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7517,"num_read_kb":36655,"num_write":459,"num_write_kb":15039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'703","reported_seq":5090,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773133+0000","last_change":"2024-09-21T23:12:59.613161+0000","last_active":"2024-09-21T23:13:18.773133+0000","last_peered":"2024-09-21T23:13:18.773133+0000","last_clean":"2024-09-21T23:13:18.773133+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:13:18.773133+0000","last_undegraded":"2024-09-21T23:13:18.773133+0000","last_fullsized":"2024-09-21T23:13:18.773133+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:56.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024929100000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5571,"num_read_kb":14116,"num_write":826,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895381+0000","last_change":"2024-09-21T23:13:03.807826+0000","last_active":"2024-09-21T23:13:18.895381+0000","last_peered":"2024-09-21T23:13:18.895381+0000","last_clean":"2024-09-21T23:13:18.895381+0000","last_became_active":"2024-09-21T23:13:03.806956+0000","last_became_peered":"2024-09-21T23:13:03.806956+0000","last_unstale":"2024-09-21T23:13:18.895381+0000","last_undegraded":"2024-09-21T23:13:18.895381+0000","last_fullsized":"2024-09-21T23:13:18.895381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:13.318790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'617","reported_seq":2796,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894136+0000","last_change":"2024-09-21T23:13:05.416619+0000","last_active":"2024-09-21T23:13:18.894136+0000","last_peered":"2024-09-21T23:13:18.894136+0000","last_clean":"2024-09-21T23:13:18.894136+0000","last_became_active":"2024-09-21T23:13:05.415035+0000","last_became_peered":"2024-09-21T23:13:05.415035+0000","last_unstale":"2024-09-21T23:13:18.894136+0000","last_undegraded":"2024-09-21T23:13:18.894136+0000","last_fullsized":"2024-09-21T23:13:18.894136+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:58.313765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043940419999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3092,"num_read_kb":25059,"num_write":602,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":379,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893977+0000","last_change":"2024-09-21T23:13:03.643721+0000","last_active":"2024-09-21T23:13:18.893977+0000","last_peered":"2024-09-21T23:13:18.893977+0000","last_clean":"2024-09-21T23:13:18.893977+0000","last_became_active":"2024-09-21T23:13:03.643364+0000","last_became_peered":"2024-09-21T23:13:03.643364+0000","last_unstale":"2024-09-21T23:13:18.893977+0000","last_undegraded":"2024-09-21T23:13:18.893977+0000","last_fullsized":"2024-09-21T23:13:18.893977+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:36.821357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"178'773","reported_seq":5979,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776076+0000","last_change":"2024-09-21T23:12:59.620332+0000","last_active":"2024-09-21T23:13:18.776076+0000","last_peered":"2024-09-21T23:13:18.776076+0000","last_clean":"2024-09-21T23:13:18.776076+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:13:18.776076+0000","last_undegraded":"2024-09-21T23:13:18.776076+0000","last_fullsized":"2024-09-21T23:13:18.776076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:08.127063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086643399999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8038,"num_read_kb":30164,"num_write":1035,"num_write_kb":19449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894851+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:13:18.894851+0000","last_peered":"2024-09-21T23:13:18.894851+0000","last_clean":"2024-09-21T23:13:18.894851+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:13:18.894851+0000","last_undegraded":"2024-09-21T23:13:18.894851+0000","last_fullsized":"2024-09-21T23:13:18.894851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"172'667","reported_seq":4018,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893811+0000","last_change":"2024-09-21T23:12:59.620148+0000","last_active":"2024-09-21T23:13:18.893811+0000","last_peered":"2024-09-21T23:13:18.893811+0000","last_clean":"2024-09-21T23:13:18.893811+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:13:18.893811+0000","last_undegraded":"2024-09-21T23:13:18.893811+0000","last_fullsized":"2024-09-21T23:13:18.893811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:04.503101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043881010000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5200,"num_read_kb":24165,"num_write":600,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894080+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:13:18.894080+0000","last_peered":"2024-09-21T23:13:18.894080+0000","last_clean":"2024-09-21T23:13:18.894080+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:13:18.894080+0000","last_undegraded":"2024-09-21T23:13:18.894080+0000","last_fullsized":"2024-09-21T23:13:18.894080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"173'829","reported_seq":6083,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894254+0000","last_change":"2024-09-21T23:12:59.623402+0000","last_active":"2024-09-21T23:13:18.894254+0000","last_peered":"2024-09-21T23:13:18.894254+0000","last_clean":"2024-09-21T23:13:18.894254+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:13:18.894254+0000","last_undegraded":"2024-09-21T23:13:18.894254+0000","last_fullsized":"2024-09-21T23:13:18.894254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:07:48.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079884020000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6855,"num_read_kb":41392,"num_write":1124,"num_write_kb":18322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'833","reported_seq":4645,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773048+0000","last_change":"2024-09-21T23:12:59.625500+0000","last_active":"2024-09-21T23:13:18.773048+0000","last_peered":"2024-09-21T23:13:18.773048+0000","last_clean":"2024-09-21T23:13:18.773048+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:13:18.773048+0000","last_undegraded":"2024-09-21T23:13:18.773048+0000","last_fullsized":"2024-09-21T23:13:18.773048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:56.312310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014050681000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5872,"num_read_kb":26777,"num_write":811,"num_write_kb":12147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":372,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893953+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:13:18.893953+0000","last_peered":"2024-09-21T23:13:18.893953+0000","last_clean":"2024-09-21T23:13:18.893953+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:13:18.893953+0000","last_undegraded":"2024-09-21T23:13:18.893953+0000","last_fullsized":"2024-09-21T23:13:18.893953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'571","reported_seq":4717,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893469+0000","last_change":"2024-09-21T23:12:59.622124+0000","last_active":"2024-09-21T23:13:18.893469+0000","last_peered":"2024-09-21T23:13:18.893469+0000","last_clean":"2024-09-21T23:13:18.893469+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:13:18.893469+0000","last_undegraded":"2024-09-21T23:13:18.893469+0000","last_fullsized":"2024-09-21T23:13:18.893469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:02.058399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061931130000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6676,"num_read_kb":20540,"num_write":495,"num_write_kb":13141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.774496+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:13:18.774496+0000","last_peered":"2024-09-21T23:13:18.774496+0000","last_clean":"2024-09-21T23:13:18.774496+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:13:18.774496+0000","last_undegraded":"2024-09-21T23:13:18.774496+0000","last_fullsized":"2024-09-21T23:13:18.774496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"177'464","reported_seq":5702,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893865+0000","last_change":"2024-09-21T23:13:05.423253+0000","last_active":"2024-09-21T23:13:18.893865+0000","last_peered":"2024-09-21T23:13:18.893865+0000","last_clean":"2024-09-21T23:13:18.893865+0000","last_became_active":"2024-09-21T23:13:05.421575+0000","last_became_peered":"2024-09-21T23:13:05.421575+0000","last_unstale":"2024-09-21T23:13:18.893865+0000","last_undegraded":"2024-09-21T23:13:18.893865+0000","last_fullsized":"2024-09-21T23:13:18.893865+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:50.035868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013442716,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4762,"num_read_kb":20313,"num_write":497,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"178'583","reported_seq":4474,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775508+0000","last_change":"2024-09-21T23:12:59.618427+0000","last_active":"2024-09-21T23:13:18.775508+0000","last_peered":"2024-09-21T23:13:18.775508+0000","last_clean":"2024-09-21T23:13:18.775508+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:13:18.775508+0000","last_undegraded":"2024-09-21T23:13:18.775508+0000","last_fullsized":"2024-09-21T23:13:18.775508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:08:21.796460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070044759999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5321,"num_read_kb":19543,"num_write":664,"num_write_kb":14858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":371,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893997+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:13:18.893997+0000","last_peered":"2024-09-21T23:13:18.893997+0000","last_clean":"2024-09-21T23:13:18.893997+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:13:18.893997+0000","last_undegraded":"2024-09-21T23:13:18.893997+0000","last_fullsized":"2024-09-21T23:13:18.893997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":470,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893940+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:13:18.893940+0000","last_peered":"2024-09-21T23:13:18.893940+0000","last_clean":"2024-09-21T23:13:18.893940+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:13:18.893940+0000","last_undegraded":"2024-09-21T23:13:18.893940+0000","last_fullsized":"2024-09-21T23:13:18.893940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'543","reported_seq":3852,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775721+0000","last_change":"2024-09-21T23:12:59.615280+0000","last_active":"2024-09-21T23:13:18.775721+0000","last_peered":"2024-09-21T23:13:18.775721+0000","last_clean":"2024-09-21T23:13:18.775721+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:13:18.775721+0000","last_undegraded":"2024-09-21T23:13:18.775721+0000","last_fullsized":"2024-09-21T23:13:18.775721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:55:07.747469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040489300000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4907,"num_read_kb":10236,"num_write":527,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"174'562","reported_seq":4608,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772591+0000","last_change":"2024-09-21T23:12:59.616991+0000","last_active":"2024-09-21T23:13:18.772591+0000","last_peered":"2024-09-21T23:13:18.772591+0000","last_clean":"2024-09-21T23:13:18.772591+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:13:18.772591+0000","last_undegraded":"2024-09-21T23:13:18.772591+0000","last_fullsized":"2024-09-21T23:13:18.772591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:16.424894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062271990000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6420,"num_read_kb":19858,"num_write":506,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"177'650","reported_seq":6686,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895221+0000","last_change":"2024-09-21T23:13:07.427624+0000","last_active":"2024-09-21T23:13:18.895221+0000","last_peered":"2024-09-21T23:13:18.895221+0000","last_clean":"2024-09-21T23:13:18.895221+0000","last_became_active":"2024-09-21T23:13:07.423010+0000","last_became_peered":"2024-09-21T23:13:07.423010+0000","last_unstale":"2024-09-21T23:13:18.895221+0000","last_undegraded":"2024-09-21T23:13:18.895221+0000","last_fullsized":"2024-09-21T23:13:18.895221+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:00.226294+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059613299999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":6967,"num_read_kb":40354,"num_write":560,"num_write_kb":11935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"174'417","reported_seq":3793,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775569+0000","last_change":"2024-09-21T23:12:59.620627+0000","last_active":"2024-09-21T23:13:18.775569+0000","last_peered":"2024-09-21T23:13:18.775569+0000","last_clean":"2024-09-21T23:13:18.775569+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:13:18.775569+0000","last_undegraded":"2024-09-21T23:13:18.775569+0000","last_fullsized":"2024-09-21T23:13:18.775569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:19.532943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009141883,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3542,"num_read_kb":26420,"num_write":356,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'558","reported_seq":2492,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894778+0000","last_change":"2024-09-21T23:12:59.620407+0000","last_active":"2024-09-21T23:13:18.894778+0000","last_peered":"2024-09-21T23:13:18.894778+0000","last_clean":"2024-09-21T23:13:18.894778+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:13:18.894778+0000","last_undegraded":"2024-09-21T23:13:18.894778+0000","last_fullsized":"2024-09-21T23:13:18.894778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:14.289431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051982479999999999,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2495,"num_read_kb":17923,"num_write":507,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'537","reported_seq":5293,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894362+0000","last_change":"2024-09-21T23:12:59.623819+0000","last_active":"2024-09-21T23:13:18.894362+0000","last_peered":"2024-09-21T23:13:18.894362+0000","last_clean":"2024-09-21T23:13:18.894362+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:13:18.894362+0000","last_undegraded":"2024-09-21T23:13:18.894362+0000","last_fullsized":"2024-09-21T23:13:18.894362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:17.779188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004612685,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5087,"num_read_kb":28815,"num_write":438,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"174'511","reported_seq":5210,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894486+0000","last_change":"2024-09-21T23:12:59.622033+0000","last_active":"2024-09-21T23:13:18.894486+0000","last_peered":"2024-09-21T23:13:18.894486+0000","last_clean":"2024-09-21T23:13:18.894486+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:13:18.894486+0000","last_undegraded":"2024-09-21T23:13:18.894486+0000","last_fullsized":"2024-09-21T23:13:18.894486+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:26.432233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033040949999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6927,"num_read_kb":23882,"num_write":481,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"173'593","reported_seq":2676,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894500+0000","last_change":"2024-09-21T23:12:59.621649+0000","last_active":"2024-09-21T23:13:18.894500+0000","last_peered":"2024-09-21T23:13:18.894500+0000","last_clean":"2024-09-21T23:13:18.894500+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:13:18.894500+0000","last_undegraded":"2024-09-21T23:13:18.894500+0000","last_fullsized":"2024-09-21T23:13:18.894500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:34.711481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027323970000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2413,"num_read_kb":21711,"num_write":633,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'671","reported_seq":4520,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895178+0000","last_change":"2024-09-21T23:13:06.426028+0000","last_active":"2024-09-21T23:13:18.895178+0000","last_peered":"2024-09-21T23:13:18.895178+0000","last_clean":"2024-09-21T23:13:18.895178+0000","last_became_active":"2024-09-21T23:13:06.420853+0000","last_became_peered":"2024-09-21T23:13:06.420853+0000","last_unstale":"2024-09-21T23:13:18.895178+0000","last_undegraded":"2024-09-21T23:13:18.895178+0000","last_fullsized":"2024-09-21T23:13:18.895178+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:29.389023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062394340000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6106,"num_read_kb":37597,"num_write":682,"num_write_kb":13471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"172'641","reported_seq":7006,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:12:59.618210+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:43.253945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011826829999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":7995,"num_read_kb":42900,"num_write":666,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'627","reported_seq":7307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893948+0000","last_change":"2024-09-21T23:13:05.416690+0000","last_active":"2024-09-21T23:13:18.893948+0000","last_peered":"2024-09-21T23:13:18.893948+0000","last_clean":"2024-09-21T23:13:18.893948+0000","last_became_active":"2024-09-21T23:13:05.415127+0000","last_became_peered":"2024-09-21T23:13:05.415127+0000","last_unstale":"2024-09-21T23:13:18.893948+0000","last_undegraded":"2024-09-21T23:13:18.893948+0000","last_fullsized":"2024-09-21T23:13:18.893948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:27.939279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032033349999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8199,"num_read_kb":39748,"num_write":659,"num_write_kb":21750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'413","reported_seq":2638,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894555+0000","last_change":"2024-09-21T23:12:59.620277+0000","last_active":"2024-09-21T23:13:18.894555+0000","last_peered":"2024-09-21T23:13:18.894555+0000","last_clean":"2024-09-21T23:13:18.894555+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:13:18.894555+0000","last_undegraded":"2024-09-21T23:13:18.894555+0000","last_fullsized":"2024-09-21T23:13:18.894555+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:07.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562690000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":1457,"num_read_kb":14145,"num_write":378,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":720,"num_read":186161,"num_read_kb":840774,"num_write":22415,"num_write_kb":399960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5301,"num_bytes_recovered":254407838,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":20934,"ondisk_log_size":20934,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29348,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342492,"statfs":{"total":96636764160,"available":96606711808,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743454,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43424,"kb_used_data":2316,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94328416,"statfs":{"total":96636764160,"available":96592297984,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77916,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293924,"statfs":{"total":96636764160,"available":96556978176,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29340,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342500,"statfs":{"total":96636764160,"available":96606720000,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55716,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316124,"statfs":{"total":96636764160,"available":96579710976,"internally_reserved":0,"allocated":2375680,"data_stored":1149820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:13:23.428 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:13:23.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:13:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:23.707 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:13:23.724 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-21T23:13:22.548291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":728,"num_whiteouts":720,"num_read":186277,"num_read_kb":840874,"num_write":22576,"num_write_kb":402378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5305,"num_bytes_recovered":255588574,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21054,"ondisk_log_size":21054,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":390164,"kb_used_data":19672,"kb_used_omap":25,"kb_used_meta":370278,"kb_avail":754584556,"statfs":{"total":773094113280,"available":772694585344,"internally_reserved":0,"allocated":20144128,"data_stored":10379068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":379164719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001805"},"pg_stats":[{"pgid":"3.1f","version":"178'580","reported_seq":4220,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894261+0000","last_change":"2024-09-21T23:13:05.425494+0000","last_active":"2024-09-21T23:13:18.894261+0000","last_peered":"2024-09-21T23:13:18.894261+0000","last_clean":"2024-09-21T23:13:18.894261+0000","last_became_active":"2024-09-21T23:13:05.423231+0000","last_became_peered":"2024-09-21T23:13:05.423231+0000","last_unstale":"2024-09-21T23:13:18.894261+0000","last_undegraded":"2024-09-21T23:13:18.894261+0000","last_fullsized":"2024-09-21T23:13:18.894261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:12:08.608124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087587719999999997,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6355,"num_read_kb":19114,"num_write":548,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"172'894","reported_seq":4765,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776370+0000","last_change":"2024-09-21T23:13:07.423842+0000","last_active":"2024-09-21T23:13:18.776370+0000","last_peered":"2024-09-21T23:13:18.776370+0000","last_clean":"2024-09-21T23:13:18.776370+0000","last_became_active":"2024-09-21T23:13:07.421837+0000","last_became_peered":"2024-09-21T23:13:07.421837+0000","last_unstale":"2024-09-21T23:13:18.776370+0000","last_undegraded":"2024-09-21T23:13:18.776370+0000","last_fullsized":"2024-09-21T23:13:18.776370+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:26.907455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001882955,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5630,"num_read_kb":41276,"num_write":1148,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'456","reported_seq":3457,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894172+0000","last_change":"2024-09-21T23:12:59.616455+0000","last_active":"2024-09-21T23:13:18.894172+0000","last_peered":"2024-09-21T23:13:18.894172+0000","last_clean":"2024-09-21T23:13:18.894172+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:13:18.894172+0000","last_undegraded":"2024-09-21T23:13:18.894172+0000","last_fullsized":"2024-09-21T23:13:18.894172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:55.115242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158018,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3362,"num_read_kb":13834,"num_write":377,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'500","reported_seq":5839,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894768+0000","last_change":"2024-09-21T23:12:59.618158+0000","last_active":"2024-09-21T23:13:18.894768+0000","last_peered":"2024-09-21T23:13:18.894768+0000","last_clean":"2024-09-21T23:13:18.894768+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:13:18.894768+0000","last_undegraded":"2024-09-21T23:13:18.894768+0000","last_fullsized":"2024-09-21T23:13:18.894768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:16.558115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037616780000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5378,"num_read_kb":22578,"num_write":533,"num_write_kb":12630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"172'618","reported_seq":4487,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894025+0000","last_change":"2024-09-21T23:13:05.423467+0000","last_active":"2024-09-21T23:13:18.894025+0000","last_peered":"2024-09-21T23:13:18.894025+0000","last_clean":"2024-09-21T23:13:18.894025+0000","last_became_active":"2024-09-21T23:13:05.421427+0000","last_became_peered":"2024-09-21T23:13:05.421427+0000","last_unstale":"2024-09-21T23:13:18.894025+0000","last_undegraded":"2024-09-21T23:13:18.894025+0000","last_fullsized":"2024-09-21T23:13:18.894025+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:42.645544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010738146,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5513,"num_read_kb":28043,"num_write":694,"num_write_kb":17938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'1323","reported_seq":7922,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772720+0000","last_change":"2024-09-21T23:12:59.615466+0000","last_active":"2024-09-21T23:13:18.772720+0000","last_peered":"2024-09-21T23:13:18.772720+0000","last_clean":"2024-09-21T23:13:18.772720+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:13:18.772720+0000","last_undegraded":"2024-09-21T23:13:18.772720+0000","last_fullsized":"2024-09-21T23:13:18.772720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:14.383756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046765799999999996,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":12289,"num_read_kb":29931,"num_write":1690,"num_write_kb":9650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"172'1009","reported_seq":3157,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893498+0000","last_change":"2024-09-21T23:12:59.622668+0000","last_active":"2024-09-21T23:13:18.893498+0000","last_peered":"2024-09-21T23:13:18.893498+0000","last_clean":"2024-09-21T23:13:18.893498+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:13:18.893498+0000","last_undegraded":"2024-09-21T23:13:18.893498+0000","last_fullsized":"2024-09-21T23:13:18.893498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:39.668516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006449682,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6326,"num_read_kb":21971,"num_write":1459,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"172'1057","reported_seq":5547,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775421+0000","last_change":"2024-09-21T23:12:59.613494+0000","last_active":"2024-09-21T23:13:18.775421+0000","last_peered":"2024-09-21T23:13:18.775421+0000","last_clean":"2024-09-21T23:13:18.775421+0000","last_became_active":"2024-09-21T23:12:37.444303+0000","last_became_peered":"2024-09-21T23:12:37.444303+0000","last_unstale":"2024-09-21T23:13:18.775421+0000","last_undegraded":"2024-09-21T23:13:18.775421+0000","last_fullsized":"2024-09-21T23:13:18.775421+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:05.804247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030213219999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8120,"num_read_kb":28946,"num_write":1438,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5999502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'652","reported_seq":5692,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894396+0000","last_change":"2024-09-21T23:13:05.425470+0000","last_active":"2024-09-21T23:13:18.894396+0000","last_peered":"2024-09-21T23:13:18.894396+0000","last_clean":"2024-09-21T23:13:18.894396+0000","last_became_active":"2024-09-21T23:13:05.423133+0000","last_became_peered":"2024-09-21T23:13:05.423133+0000","last_unstale":"2024-09-21T23:13:18.894396+0000","last_undegraded":"2024-09-21T23:13:18.894396+0000","last_fullsized":"2024-09-21T23:13:18.894396+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:38.996927+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022352600000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":6037,"num_read_kb":23064,"num_write":602,"num_write_kb":12198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'521","reported_seq":4585,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894097+0000","last_change":"2024-09-21T23:12:59.621765+0000","last_active":"2024-09-21T23:13:18.894097+0000","last_peered":"2024-09-21T23:13:18.894097+0000","last_clean":"2024-09-21T23:13:18.894097+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:13:18.894097+0000","last_undegraded":"2024-09-21T23:13:18.894097+0000","last_fullsized":"2024-09-21T23:13:18.894097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:11.726440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066845790000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5732,"num_read_kb":29704,"num_write":420,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"178'561","reported_seq":6978,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894886+0000","last_change":"2024-09-21T23:12:59.624274+0000","last_active":"2024-09-21T23:13:18.894886+0000","last_peered":"2024-09-21T23:13:18.894886+0000","last_clean":"2024-09-21T23:13:18.894886+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:13:18.894886+0000","last_undegraded":"2024-09-21T23:13:18.894886+0000","last_fullsized":"2024-09-21T23:13:18.894886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:35.264209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048388729999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7517,"num_read_kb":36655,"num_write":459,"num_write_kb":15039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'703","reported_seq":5090,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773133+0000","last_change":"2024-09-21T23:12:59.613161+0000","last_active":"2024-09-21T23:13:18.773133+0000","last_peered":"2024-09-21T23:13:18.773133+0000","last_clean":"2024-09-21T23:13:18.773133+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:13:18.773133+0000","last_undegraded":"2024-09-21T23:13:18.773133+0000","last_fullsized":"2024-09-21T23:13:18.773133+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:56.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024929100000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5571,"num_read_kb":14116,"num_write":826,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895381+0000","last_change":"2024-09-21T23:13:03.807826+0000","last_active":"2024-09-21T23:13:18.895381+0000","last_peered":"2024-09-21T23:13:18.895381+0000","last_clean":"2024-09-21T23:13:18.895381+0000","last_became_active":"2024-09-21T23:13:03.806956+0000","last_became_peered":"2024-09-21T23:13:03.806956+0000","last_unstale":"2024-09-21T23:13:18.895381+0000","last_undegraded":"2024-09-21T23:13:18.895381+0000","last_fullsized":"2024-09-21T23:13:18.895381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:13.318790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'617","reported_seq":2796,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894136+0000","last_change":"2024-09-21T23:13:05.416619+0000","last_active":"2024-09-21T23:13:18.894136+0000","last_peered":"2024-09-21T23:13:18.894136+0000","last_clean":"2024-09-21T23:13:18.894136+0000","last_became_active":"2024-09-21T23:13:05.415035+0000","last_became_peered":"2024-09-21T23:13:05.415035+0000","last_unstale":"2024-09-21T23:13:18.894136+0000","last_undegraded":"2024-09-21T23:13:18.894136+0000","last_fullsized":"2024-09-21T23:13:18.894136+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:14:58.313765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043940419999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3092,"num_read_kb":25059,"num_write":602,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":379,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893977+0000","last_change":"2024-09-21T23:13:03.643721+0000","last_active":"2024-09-21T23:13:18.893977+0000","last_peered":"2024-09-21T23:13:18.893977+0000","last_clean":"2024-09-21T23:13:18.893977+0000","last_became_active":"2024-09-21T23:13:03.643364+0000","last_became_peered":"2024-09-21T23:13:03.643364+0000","last_unstale":"2024-09-21T23:13:18.893977+0000","last_undegraded":"2024-09-21T23:13:18.893977+0000","last_fullsized":"2024-09-21T23:13:18.893977+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:36.821357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"178'773","reported_seq":5979,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776076+0000","last_change":"2024-09-21T23:12:59.620332+0000","last_active":"2024-09-21T23:13:18.776076+0000","last_peered":"2024-09-21T23:13:18.776076+0000","last_clean":"2024-09-21T23:13:18.776076+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:13:18.776076+0000","last_undegraded":"2024-09-21T23:13:18.776076+0000","last_fullsized":"2024-09-21T23:13:18.776076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:08.127063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086643399999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8038,"num_read_kb":30164,"num_write":1035,"num_write_kb":19449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894851+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:13:18.894851+0000","last_peered":"2024-09-21T23:13:18.894851+0000","last_clean":"2024-09-21T23:13:18.894851+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:13:18.894851+0000","last_undegraded":"2024-09-21T23:13:18.894851+0000","last_fullsized":"2024-09-21T23:13:18.894851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"172'667","reported_seq":4018,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893811+0000","last_change":"2024-09-21T23:12:59.620148+0000","last_active":"2024-09-21T23:13:18.893811+0000","last_peered":"2024-09-21T23:13:18.893811+0000","last_clean":"2024-09-21T23:13:18.893811+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:13:18.893811+0000","last_undegraded":"2024-09-21T23:13:18.893811+0000","last_fullsized":"2024-09-21T23:13:18.893811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:04.503101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043881010000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5200,"num_read_kb":24165,"num_write":600,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894080+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:13:18.894080+0000","last_peered":"2024-09-21T23:13:18.894080+0000","last_clean":"2024-09-21T23:13:18.894080+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:13:18.894080+0000","last_undegraded":"2024-09-21T23:13:18.894080+0000","last_fullsized":"2024-09-21T23:13:18.894080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"173'829","reported_seq":6083,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894254+0000","last_change":"2024-09-21T23:12:59.623402+0000","last_active":"2024-09-21T23:13:18.894254+0000","last_peered":"2024-09-21T23:13:18.894254+0000","last_clean":"2024-09-21T23:13:18.894254+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:13:18.894254+0000","last_undegraded":"2024-09-21T23:13:18.894254+0000","last_fullsized":"2024-09-21T23:13:18.894254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:07:48.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079884020000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6855,"num_read_kb":41392,"num_write":1124,"num_write_kb":18322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'833","reported_seq":4645,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773048+0000","last_change":"2024-09-21T23:12:59.625500+0000","last_active":"2024-09-21T23:13:18.773048+0000","last_peered":"2024-09-21T23:13:18.773048+0000","last_clean":"2024-09-21T23:13:18.773048+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:13:18.773048+0000","last_undegraded":"2024-09-21T23:13:18.773048+0000","last_fullsized":"2024-09-21T23:13:18.773048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:56.312310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014050681000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5872,"num_read_kb":26777,"num_write":811,"num_write_kb":12147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":372,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893953+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:13:18.893953+0000","last_peered":"2024-09-21T23:13:18.893953+0000","last_clean":"2024-09-21T23:13:18.893953+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:13:18.893953+0000","last_undegraded":"2024-09-21T23:13:18.893953+0000","last_fullsized":"2024-09-21T23:13:18.893953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'571","reported_seq":4717,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893469+0000","last_change":"2024-09-21T23:12:59.622124+0000","last_active":"2024-09-21T23:13:18.893469+0000","last_peered":"2024-09-21T23:13:18.893469+0000","last_clean":"2024-09-21T23:13:18.893469+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:13:18.893469+0000","last_undegraded":"2024-09-21T23:13:18.893469+0000","last_fullsized":"2024-09-21T23:13:18.893469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:02.058399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061931130000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6676,"num_read_kb":20540,"num_write":495,"num_write_kb":13141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.774496+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:13:18.774496+0000","last_peered":"2024-09-21T23:13:18.774496+0000","last_clean":"2024-09-21T23:13:18.774496+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:13:18.774496+0000","last_undegraded":"2024-09-21T23:13:18.774496+0000","last_fullsized":"2024-09-21T23:13:18.774496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"177'464","reported_seq":5702,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893865+0000","last_change":"2024-09-21T23:13:05.423253+0000","last_active":"2024-09-21T23:13:18.893865+0000","last_peered":"2024-09-21T23:13:18.893865+0000","last_clean":"2024-09-21T23:13:18.893865+0000","last_became_active":"2024-09-21T23:13:05.421575+0000","last_became_peered":"2024-09-21T23:13:05.421575+0000","last_unstale":"2024-09-21T23:13:18.893865+0000","last_undegraded":"2024-09-21T23:13:18.893865+0000","last_fullsized":"2024-09-21T23:13:18.893865+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:50.035868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013442716,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4762,"num_read_kb":20313,"num_write":497,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"178'583","reported_seq":4474,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775508+0000","last_change":"2024-09-21T23:12:59.618427+0000","last_active":"2024-09-21T23:13:18.775508+0000","last_peered":"2024-09-21T23:13:18.775508+0000","last_clean":"2024-09-21T23:13:18.775508+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:13:18.775508+0000","last_undegraded":"2024-09-21T23:13:18.775508+0000","last_fullsized":"2024-09-21T23:13:18.775508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:08:21.796460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070044759999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5321,"num_read_kb":19543,"num_write":664,"num_write_kb":14858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":371,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893997+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:13:18.893997+0000","last_peered":"2024-09-21T23:13:18.893997+0000","last_clean":"2024-09-21T23:13:18.893997+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:13:18.893997+0000","last_undegraded":"2024-09-21T23:13:18.893997+0000","last_fullsized":"2024-09-21T23:13:18.893997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":470,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893940+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:13:18.893940+0000","last_peered":"2024-09-21T23:13:18.893940+0000","last_clean":"2024-09-21T23:13:18.893940+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:13:18.893940+0000","last_undegraded":"2024-09-21T23:13:18.893940+0000","last_fullsized":"2024-09-21T23:13:18.893940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'543","reported_seq":3852,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775721+0000","last_change":"2024-09-21T23:12:59.615280+0000","last_active":"2024-09-21T23:13:18.775721+0000","last_peered":"2024-09-21T23:13:18.775721+0000","last_clean":"2024-09-21T23:13:18.775721+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:13:18.775721+0000","last_undegraded":"2024-09-21T23:13:18.775721+0000","last_fullsized":"2024-09-21T23:13:18.775721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:55:07.747469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040489300000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4907,"num_read_kb":10236,"num_write":527,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"174'562","reported_seq":4608,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772591+0000","last_change":"2024-09-21T23:12:59.616991+0000","last_active":"2024-09-21T23:13:18.772591+0000","last_peered":"2024-09-21T23:13:18.772591+0000","last_clean":"2024-09-21T23:13:18.772591+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:13:18.772591+0000","last_undegraded":"2024-09-21T23:13:18.772591+0000","last_fullsized":"2024-09-21T23:13:18.772591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:16.424894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062271990000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6420,"num_read_kb":19858,"num_write":506,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"177'650","reported_seq":6686,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895221+0000","last_change":"2024-09-21T23:13:07.427624+0000","last_active":"2024-09-21T23:13:18.895221+0000","last_peered":"2024-09-21T23:13:18.895221+0000","last_clean":"2024-09-21T23:13:18.895221+0000","last_became_active":"2024-09-21T23:13:07.423010+0000","last_became_peered":"2024-09-21T23:13:07.423010+0000","last_unstale":"2024-09-21T23:13:18.895221+0000","last_undegraded":"2024-09-21T23:13:18.895221+0000","last_fullsized":"2024-09-21T23:13:18.895221+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:00.226294+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059613299999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":6967,"num_read_kb":40354,"num_write":560,"num_write_kb":11935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"174'417","reported_seq":3793,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775569+0000","last_change":"2024-09-21T23:12:59.620627+0000","last_active":"2024-09-21T23:13:18.775569+0000","last_peered":"2024-09-21T23:13:18.775569+0000","last_clean":"2024-09-21T23:13:18.775569+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:13:18.775569+0000","last_undegraded":"2024-09-21T23:13:18.775569+0000","last_fullsized":"2024-09-21T23:13:18.775569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:19.532943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009141883,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3542,"num_read_kb":26420,"num_write":356,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'558","reported_seq":2492,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894778+0000","last_change":"2024-09-21T23:12:59.620407+0000","last_active":"2024-09-21T23:13:18.894778+0000","last_peered":"2024-09-21T23:13:18.894778+0000","last_clean":"2024-09-21T23:13:18.894778+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:13:18.894778+0000","last_undegraded":"2024-09-21T23:13:18.894778+0000","last_fullsized":"2024-09-21T23:13:18.894778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:14.289431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051982479999999999,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2495,"num_read_kb":17923,"num_write":507,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'537","reported_seq":5293,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894362+0000","last_change":"2024-09-21T23:12:59.623819+0000","last_active":"2024-09-21T23:13:18.894362+0000","last_peered":"2024-09-21T23:13:18.894362+0000","last_clean":"2024-09-21T23:13:18.894362+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:13:18.894362+0000","last_undegraded":"2024-09-21T23:13:18.894362+0000","last_fullsized":"2024-09-21T23:13:18.894362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:17.779188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004612685,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5087,"num_read_kb":28815,"num_write":438,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"174'511","reported_seq":5210,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894486+0000","last_change":"2024-09-21T23:12:59.622033+0000","last_active":"2024-09-21T23:13:18.894486+0000","last_peered":"2024-09-21T23:13:18.894486+0000","last_clean":"2024-09-21T23:13:18.894486+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:13:18.894486+0000","last_undegraded":"2024-09-21T23:13:18.894486+0000","last_fullsized":"2024-09-21T23:13:18.894486+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:26.432233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033040949999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6927,"num_read_kb":23882,"num_write":481,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"173'593","reported_seq":2676,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894500+0000","last_change":"2024-09-21T23:12:59.621649+0000","last_active":"2024-09-21T23:13:18.894500+0000","last_peered":"2024-09-21T23:13:18.894500+0000","last_clean":"2024-09-21T23:13:18.894500+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:13:18.894500+0000","last_undegraded":"2024-09-21T23:13:18.894500+0000","last_fullsized":"2024-09-21T23:13:18.894500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:34.711481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027323970000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2413,"num_read_kb":21711,"num_write":633,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'671","reported_seq":4520,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895178+0000","last_change":"2024-09-21T23:13:06.426028+0000","last_active":"2024-09-21T23:13:18.895178+0000","last_peered":"2024-09-21T23:13:18.895178+0000","last_clean":"2024-09-21T23:13:18.895178+0000","last_became_active":"2024-09-21T23:13:06.420853+0000","last_became_peered":"2024-09-21T23:13:06.420853+0000","last_unstale":"2024-09-21T23:13:18.895178+0000","last_undegraded":"2024-09-21T23:13:18.895178+0000","last_fullsized":"2024-09-21T23:13:18.895178+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:29.389023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062394340000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6106,"num_read_kb":37597,"num_write":682,"num_write_kb":13471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"172'641","reported_seq":7006,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:12:59.618210+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:43.253945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011826829999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":7995,"num_read_kb":42900,"num_write":666,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'627","reported_seq":7307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893948+0000","last_change":"2024-09-21T23:13:05.416690+0000","last_active":"2024-09-21T23:13:18.893948+0000","last_peered":"2024-09-21T23:13:18.893948+0000","last_clean":"2024-09-21T23:13:18.893948+0000","last_became_active":"2024-09-21T23:13:05.415127+0000","last_became_peered":"2024-09-21T23:13:05.415127+0000","last_unstale":"2024-09-21T23:13:18.893948+0000","last_undegraded":"2024-09-21T23:13:18.893948+0000","last_fullsized":"2024-09-21T23:13:18.893948+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:27.939279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032033349999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8199,"num_read_kb":39748,"num_write":659,"num_write_kb":21750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'413","reported_seq":2638,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894555+0000","last_change":"2024-09-21T23:12:59.620277+0000","last_active":"2024-09-21T23:13:18.894555+0000","last_peered":"2024-09-21T23:13:18.894555+0000","last_clean":"2024-09-21T23:13:18.894555+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:13:18.894555+0000","last_undegraded":"2024-09-21T23:13:18.894555+0000","last_fullsized":"2024-09-21T23:13:18.894555+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:07.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562690000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":1457,"num_read_kb":14145,"num_write":378,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":720,"num_read":186161,"num_read_kb":840774,"num_write":22415,"num_write_kb":399960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5301,"num_bytes_recovered":254407838,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":20934,"ondisk_log_size":20934,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29348,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342492,"statfs":{"total":96636764160,"available":96606711808,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743454,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43424,"kb_used_data":2316,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94328416,"statfs":{"total":96636764160,"available":96592297984,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77916,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293924,"statfs":{"total":96636764160,"available":96556978176,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29340,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342500,"statfs":{"total":96636764160,"available":96606720000,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55716,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316124,"statfs":{"total":96636764160,"available":96579710976,"internally_reserved":0,"allocated":2375680,"data_stored":1149820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:13:23.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:13:24.007 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:24.007 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":196,"fsid":"9ce7cfc1-a7d6-426f-bf44-a308fb883b8f","created":"2024-09-21T23:09:25.662940+0000","modified":"2024-09-21T23:13:18.766401+0000","last_up_change":"2024-09-21T23:11:01.346212+0000","last_in_change":"2024-09-21T23:13:02.619106+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:09:40.569134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:09:44.626156+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T23:09:53.888601+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":178,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e2bdbec-1034-49cf-a4f1-a5ef1d83e04d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6817","nonce":198356244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6819","nonce":198356244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6823","nonce":198356244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":198356244},{"type":"v1","addr":"172.21.15.150:6821","nonce":198356244}]},"public_addr":"172.21.15.150:6817/198356244","cluster_addr":"172.21.15.150:6819/198356244","heartbeat_back_addr":"172.21.15.150:6823/198356244","heartbeat_front_addr":"172.21.15.150:6821/198356244","state":["exists","up"]},{"osd":1,"uuid":"5fb69564-3b15-470e-b45f-f70594751344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6825","nonce":725319294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6827","nonce":725319294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6831","nonce":725319294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":725319294},{"type":"v1","addr":"172.21.15.150:6829","nonce":725319294}]},"public_addr":"172.21.15.150:6825/725319294","cluster_addr":"172.21.15.150:6827/725319294","heartbeat_back_addr":"172.21.15.150:6831/725319294","heartbeat_front_addr":"172.21.15.150:6829/725319294","state":["exists","up"]},{"osd":2,"uuid":"cc3e3e88-a324-4e3d-8b40-0d8c93da1400","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6801","nonce":1956153436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6803","nonce":1956153436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6807","nonce":1956153436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1956153436},{"type":"v1","addr":"172.21.15.150:6805","nonce":1956153436}]},"public_addr":"172.21.15.150:6801/1956153436","cluster_addr":"172.21.15.150:6803/1956153436","heartbeat_back_addr":"172.21.15.150:6807/1956153436","heartbeat_front_addr":"172.21.15.150:6805/1956153436","state":["exists","up"]},{"osd":3,"uuid":"933fe042-8aed-4795-918c-572cf81999a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6809","nonce":3049870482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6811","nonce":3049870482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6815","nonce":3049870482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3049870482},{"type":"v1","addr":"172.21.15.150:6813","nonce":3049870482}]},"public_addr":"172.21.15.150:6809/3049870482","cluster_addr":"172.21.15.150:6811/3049870482","heartbeat_back_addr":"172.21.15.150:6815/3049870482","heartbeat_front_addr":"172.21.15.150:6813/3049870482","state":["exists","up"]},{"osd":4,"uuid":"af826961-2dad-4702-a904-991e2f5f57c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":32,"up_from":69,"up_thru":178,"down_at":33,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6801","nonce":3379790213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6803","nonce":3379790213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6807","nonce":3379790213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3379790213},{"type":"v1","addr":"172.21.15.175:6805","nonce":3379790213}]},"public_addr":"172.21.15.175:6801/3379790213","cluster_addr":"172.21.15.175:6803/3379790213","heartbeat_back_addr":"172.21.15.175:6807/3379790213","heartbeat_front_addr":"172.21.15.175:6805/3379790213","state":["exists","up"]},{"osd":5,"uuid":"c4f1908b-0885-408e-873e-4d2aa9b9c506","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6827","nonce":120048378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6829","nonce":120048378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6833","nonce":120048378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":120048378},{"type":"v1","addr":"172.21.15.175:6831","nonce":120048378}]},"public_addr":"172.21.15.175:6827/120048378","cluster_addr":"172.21.15.175:6829/120048378","heartbeat_back_addr":"172.21.15.175:6833/120048378","heartbeat_front_addr":"172.21.15.175:6831/120048378","state":["exists","up"]},{"osd":6,"uuid":"08cef493-ce38-4b52-b5f2-23e7694953c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6819","nonce":787648514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6821","nonce":787648514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6825","nonce":787648514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":787648514},{"type":"v1","addr":"172.21.15.175:6823","nonce":787648514}]},"public_addr":"172.21.15.175:6819/787648514","cluster_addr":"172.21.15.175:6821/787648514","heartbeat_back_addr":"172.21.15.175:6825/787648514","heartbeat_front_addr":"172.21.15.175:6823/787648514","state":["exists","up"]},{"osd":7,"uuid":"26d2142e-419c-4da3-9c28-ac442b1b66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6809","nonce":1424462696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6811","nonce":1424462696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6815","nonce":1424462696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1424462696},{"type":"v1","addr":"172.21.15.175:6813","nonce":1424462696}]},"public_addr":"172.21.15.175:6809/1424462696","cluster_addr":"172.21.15.175:6811/1424462696","heartbeat_back_addr":"172.21.15.175:6815/1424462696","heartbeat_front_addr":"172.21.15.175:6813/1424462696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.004700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.958497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.988302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.993971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.945648+0000","dead_epoch":33},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:36.366302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.852613+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:09:35.713697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:13:25.025 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T23:13:25.025 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:25.183 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:25.183 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:25.183 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:25.199 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T23:13:25.481 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T23:13:25.499 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T23:13:25.499 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:25.643 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:25.643 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:25.658 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T23:13:25.932 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T23:13:25.950 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T23:13:25.950 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:26.085 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:26.085 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:26.085 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:26.100 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T23:13:26.382 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T23:13:26.400 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T23:13:26.400 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:26.546 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:26.546 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:26.547 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:26.559 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T23:13:26.843 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T23:13:26.862 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T23:13:26.862 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:27.001 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:27.001 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:27.001 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:27.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T23:13:27.294 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T23:13:27.318 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T23:13:27.318 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:27.463 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:27.463 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:27.463 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:27.480 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T23:13:27.762 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T23:13:27.781 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T23:13:27.781 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:27.932 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:27.932 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:27.948 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T23:13:28.233 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T23:13:28.253 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T23:13:28.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:13:28.401 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-21T23:13:28.401 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:13:28.401 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-21T23:13:28.416 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T23:13:28.714 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T23:13:28.734 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:13:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:29.040 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:13:29.057 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-21T23:13:28.550488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":728,"num_whiteouts":720,"num_read":186277,"num_read_kb":840874,"num_write":22576,"num_write_kb":402378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5305,"num_bytes_recovered":255588574,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21054,"ondisk_log_size":21054,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":390164,"kb_used_data":19672,"kb_used_omap":25,"kb_used_meta":370278,"kb_avail":754584556,"statfs":{"total":773094113280,"available":772694585344,"internally_reserved":0,"allocated":20144128,"data_stored":10379068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":379164719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003420"},"pg_stats":[{"pgid":"3.1f","version":"178'580","reported_seq":4221,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894307+0000","last_change":"2024-09-21T23:13:05.425494+0000","last_active":"2024-09-21T23:13:18.894307+0000","last_peered":"2024-09-21T23:13:18.894307+0000","last_clean":"2024-09-21T23:13:18.894307+0000","last_became_active":"2024-09-21T23:13:05.423231+0000","last_became_peered":"2024-09-21T23:13:05.423231+0000","last_unstale":"2024-09-21T23:13:18.894307+0000","last_undegraded":"2024-09-21T23:13:18.894307+0000","last_fullsized":"2024-09-21T23:13:18.894307+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087587719999999997,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6355,"num_read_kb":19114,"num_write":548,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"172'894","reported_seq":4765,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776370+0000","last_change":"2024-09-21T23:13:07.423842+0000","last_active":"2024-09-21T23:13:18.776370+0000","last_peered":"2024-09-21T23:13:18.776370+0000","last_clean":"2024-09-21T23:13:18.776370+0000","last_became_active":"2024-09-21T23:13:07.421837+0000","last_became_peered":"2024-09-21T23:13:07.421837+0000","last_unstale":"2024-09-21T23:13:18.776370+0000","last_undegraded":"2024-09-21T23:13:18.776370+0000","last_fullsized":"2024-09-21T23:13:18.776370+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:26.907455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001882955,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5630,"num_read_kb":41276,"num_write":1148,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'456","reported_seq":3457,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894172+0000","last_change":"2024-09-21T23:12:59.616455+0000","last_active":"2024-09-21T23:13:18.894172+0000","last_peered":"2024-09-21T23:13:18.894172+0000","last_clean":"2024-09-21T23:13:18.894172+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:13:18.894172+0000","last_undegraded":"2024-09-21T23:13:18.894172+0000","last_fullsized":"2024-09-21T23:13:18.894172+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:55.115242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158018,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3362,"num_read_kb":13834,"num_write":377,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'500","reported_seq":5839,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894768+0000","last_change":"2024-09-21T23:12:59.618158+0000","last_active":"2024-09-21T23:13:18.894768+0000","last_peered":"2024-09-21T23:13:18.894768+0000","last_clean":"2024-09-21T23:13:18.894768+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:13:18.894768+0000","last_undegraded":"2024-09-21T23:13:18.894768+0000","last_fullsized":"2024-09-21T23:13:18.894768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:16.558115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037616780000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5378,"num_read_kb":22578,"num_write":533,"num_write_kb":12630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"172'618","reported_seq":4488,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894147+0000","last_change":"2024-09-21T23:13:05.423467+0000","last_active":"2024-09-21T23:13:18.894147+0000","last_peered":"2024-09-21T23:13:18.894147+0000","last_clean":"2024-09-21T23:13:18.894147+0000","last_became_active":"2024-09-21T23:13:05.421427+0000","last_became_peered":"2024-09-21T23:13:05.421427+0000","last_unstale":"2024-09-21T23:13:18.894147+0000","last_undegraded":"2024-09-21T23:13:18.894147+0000","last_fullsized":"2024-09-21T23:13:18.894147+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010738146,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5513,"num_read_kb":28043,"num_write":694,"num_write_kb":17938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'1323","reported_seq":7922,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772720+0000","last_change":"2024-09-21T23:12:59.615466+0000","last_active":"2024-09-21T23:13:18.772720+0000","last_peered":"2024-09-21T23:13:18.772720+0000","last_clean":"2024-09-21T23:13:18.772720+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:13:18.772720+0000","last_undegraded":"2024-09-21T23:13:18.772720+0000","last_fullsized":"2024-09-21T23:13:18.772720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1323,"log_dups_size":0,"ondisk_log_size":1323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:14.383756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046765799999999996,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":12289,"num_read_kb":29931,"num_write":1690,"num_write_kb":9650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"172'1009","reported_seq":3157,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893498+0000","last_change":"2024-09-21T23:12:59.622668+0000","last_active":"2024-09-21T23:13:18.893498+0000","last_peered":"2024-09-21T23:13:18.893498+0000","last_clean":"2024-09-21T23:13:18.893498+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:13:18.893498+0000","last_undegraded":"2024-09-21T23:13:18.893498+0000","last_fullsized":"2024-09-21T23:13:18.893498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:39.668516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006449682,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6326,"num_read_kb":21971,"num_write":1459,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"172'1057","reported_seq":5547,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775421+0000","last_change":"2024-09-21T23:12:59.613494+0000","last_active":"2024-09-21T23:13:18.775421+0000","last_peered":"2024-09-21T23:13:18.775421+0000","last_clean":"2024-09-21T23:13:18.775421+0000","last_became_active":"2024-09-21T23:12:37.444303+0000","last_became_peered":"2024-09-21T23:12:37.444303+0000","last_unstale":"2024-09-21T23:13:18.775421+0000","last_undegraded":"2024-09-21T23:13:18.775421+0000","last_fullsized":"2024-09-21T23:13:18.775421+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:04:05.804247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030213219999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8120,"num_read_kb":28946,"num_write":1438,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5999502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"175'652","reported_seq":5693,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894474+0000","last_change":"2024-09-21T23:13:05.425470+0000","last_active":"2024-09-21T23:13:18.894474+0000","last_peered":"2024-09-21T23:13:18.894474+0000","last_clean":"2024-09-21T23:13:18.894474+0000","last_became_active":"2024-09-21T23:13:05.423133+0000","last_became_peered":"2024-09-21T23:13:05.423133+0000","last_unstale":"2024-09-21T23:13:18.894474+0000","last_undegraded":"2024-09-21T23:13:18.894474+0000","last_fullsized":"2024-09-21T23:13:18.894474+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022352600000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":6037,"num_read_kb":23064,"num_write":602,"num_write_kb":12198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"174'521","reported_seq":4585,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894097+0000","last_change":"2024-09-21T23:12:59.621765+0000","last_active":"2024-09-21T23:13:18.894097+0000","last_peered":"2024-09-21T23:13:18.894097+0000","last_clean":"2024-09-21T23:13:18.894097+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:13:18.894097+0000","last_undegraded":"2024-09-21T23:13:18.894097+0000","last_fullsized":"2024-09-21T23:13:18.894097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:11.726440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066845790000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5732,"num_read_kb":29704,"num_write":420,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"178'561","reported_seq":6978,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894886+0000","last_change":"2024-09-21T23:12:59.624274+0000","last_active":"2024-09-21T23:13:18.894886+0000","last_peered":"2024-09-21T23:13:18.894886+0000","last_clean":"2024-09-21T23:13:18.894886+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:13:18.894886+0000","last_undegraded":"2024-09-21T23:13:18.894886+0000","last_fullsized":"2024-09-21T23:13:18.894886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:35.264209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048388729999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7517,"num_read_kb":36655,"num_write":459,"num_write_kb":15039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'703","reported_seq":5090,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773133+0000","last_change":"2024-09-21T23:12:59.613161+0000","last_active":"2024-09-21T23:13:18.773133+0000","last_peered":"2024-09-21T23:13:18.773133+0000","last_clean":"2024-09-21T23:13:18.773133+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:13:18.773133+0000","last_undegraded":"2024-09-21T23:13:18.773133+0000","last_fullsized":"2024-09-21T23:13:18.773133+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:56.299549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024929100000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5571,"num_read_kb":14116,"num_write":826,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895381+0000","last_change":"2024-09-21T23:13:03.807826+0000","last_active":"2024-09-21T23:13:18.895381+0000","last_peered":"2024-09-21T23:13:18.895381+0000","last_clean":"2024-09-21T23:13:18.895381+0000","last_became_active":"2024-09-21T23:13:03.806956+0000","last_became_peered":"2024-09-21T23:13:03.806956+0000","last_unstale":"2024-09-21T23:13:18.895381+0000","last_undegraded":"2024-09-21T23:13:18.895381+0000","last_fullsized":"2024-09-21T23:13:18.895381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:13.318790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'617","reported_seq":2797,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894186+0000","last_change":"2024-09-21T23:13:05.416619+0000","last_active":"2024-09-21T23:13:18.894186+0000","last_peered":"2024-09-21T23:13:18.894186+0000","last_clean":"2024-09-21T23:13:18.894186+0000","last_became_active":"2024-09-21T23:13:05.415035+0000","last_became_peered":"2024-09-21T23:13:05.415035+0000","last_unstale":"2024-09-21T23:13:18.894186+0000","last_undegraded":"2024-09-21T23:13:18.894186+0000","last_fullsized":"2024-09-21T23:13:18.894186+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043940419999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3092,"num_read_kb":25059,"num_write":602,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":379,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893977+0000","last_change":"2024-09-21T23:13:03.643721+0000","last_active":"2024-09-21T23:13:18.893977+0000","last_peered":"2024-09-21T23:13:18.893977+0000","last_clean":"2024-09-21T23:13:18.893977+0000","last_became_active":"2024-09-21T23:13:03.643364+0000","last_became_peered":"2024-09-21T23:13:03.643364+0000","last_unstale":"2024-09-21T23:13:18.893977+0000","last_undegraded":"2024-09-21T23:13:18.893977+0000","last_fullsized":"2024-09-21T23:13:18.893977+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:36.821357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"178'773","reported_seq":5979,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.776076+0000","last_change":"2024-09-21T23:12:59.620332+0000","last_active":"2024-09-21T23:13:18.776076+0000","last_peered":"2024-09-21T23:13:18.776076+0000","last_clean":"2024-09-21T23:13:18.776076+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:13:18.776076+0000","last_undegraded":"2024-09-21T23:13:18.776076+0000","last_fullsized":"2024-09-21T23:13:18.776076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:08.127063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086643399999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8038,"num_read_kb":30164,"num_write":1035,"num_write_kb":19449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894851+0000","last_change":"2024-09-21T23:11:11.426487+0000","last_active":"2024-09-21T23:13:18.894851+0000","last_peered":"2024-09-21T23:13:18.894851+0000","last_clean":"2024-09-21T23:13:18.894851+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:13:18.894851+0000","last_undegraded":"2024-09-21T23:13:18.894851+0000","last_fullsized":"2024-09-21T23:13:18.894851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:05.253808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"172'667","reported_seq":4018,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893811+0000","last_change":"2024-09-21T23:12:59.620148+0000","last_active":"2024-09-21T23:13:18.893811+0000","last_peered":"2024-09-21T23:13:18.893811+0000","last_clean":"2024-09-21T23:13:18.893811+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:13:18.893811+0000","last_undegraded":"2024-09-21T23:13:18.893811+0000","last_fullsized":"2024-09-21T23:13:18.893811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:04.503101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043881010000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5200,"num_read_kb":24165,"num_write":600,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894080+0000","last_change":"2024-09-21T23:09:47.726220+0000","last_active":"2024-09-21T23:13:18.894080+0000","last_peered":"2024-09-21T23:13:18.894080+0000","last_clean":"2024-09-21T23:13:18.894080+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:13:18.894080+0000","last_undegraded":"2024-09-21T23:13:18.894080+0000","last_fullsized":"2024-09-21T23:13:18.894080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:01.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"173'829","reported_seq":6083,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894254+0000","last_change":"2024-09-21T23:12:59.623402+0000","last_active":"2024-09-21T23:13:18.894254+0000","last_peered":"2024-09-21T23:13:18.894254+0000","last_clean":"2024-09-21T23:13:18.894254+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:13:18.894254+0000","last_undegraded":"2024-09-21T23:13:18.894254+0000","last_fullsized":"2024-09-21T23:13:18.894254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:07:48.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079884020000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6855,"num_read_kb":41392,"num_write":1124,"num_write_kb":18322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'833","reported_seq":4645,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.773048+0000","last_change":"2024-09-21T23:12:59.625500+0000","last_active":"2024-09-21T23:13:18.773048+0000","last_peered":"2024-09-21T23:13:18.773048+0000","last_clean":"2024-09-21T23:13:18.773048+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:13:18.773048+0000","last_undegraded":"2024-09-21T23:13:18.773048+0000","last_fullsized":"2024-09-21T23:13:18.773048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:56.312310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014050681000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5872,"num_read_kb":26777,"num_write":811,"num_write_kb":12147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":372,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893953+0000","last_change":"2024-09-21T23:09:47.724225+0000","last_active":"2024-09-21T23:13:18.893953+0000","last_peered":"2024-09-21T23:13:18.893953+0000","last_clean":"2024-09-21T23:13:18.893953+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:13:18.893953+0000","last_undegraded":"2024-09-21T23:13:18.893953+0000","last_fullsized":"2024-09-21T23:13:18.893953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:20.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'571","reported_seq":4717,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893469+0000","last_change":"2024-09-21T23:12:59.622124+0000","last_active":"2024-09-21T23:13:18.893469+0000","last_peered":"2024-09-21T23:13:18.893469+0000","last_clean":"2024-09-21T23:13:18.893469+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:13:18.893469+0000","last_undegraded":"2024-09-21T23:13:18.893469+0000","last_fullsized":"2024-09-21T23:13:18.893469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:02.058399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061931130000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6676,"num_read_kb":20540,"num_write":495,"num_write_kb":13141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":369,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.774496+0000","last_change":"2024-09-21T23:09:47.716691+0000","last_active":"2024-09-21T23:13:18.774496+0000","last_peered":"2024-09-21T23:13:18.774496+0000","last_clean":"2024-09-21T23:13:18.774496+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:13:18.774496+0000","last_undegraded":"2024-09-21T23:13:18.774496+0000","last_fullsized":"2024-09-21T23:13:18.774496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:44.148325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"177'464","reported_seq":5703,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894004+0000","last_change":"2024-09-21T23:13:05.423253+0000","last_active":"2024-09-21T23:13:18.894004+0000","last_peered":"2024-09-21T23:13:18.894004+0000","last_clean":"2024-09-21T23:13:18.894004+0000","last_became_active":"2024-09-21T23:13:05.421575+0000","last_became_peered":"2024-09-21T23:13:05.421575+0000","last_unstale":"2024-09-21T23:13:18.894004+0000","last_undegraded":"2024-09-21T23:13:18.894004+0000","last_fullsized":"2024-09-21T23:13:18.894004+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013442716,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4762,"num_read_kb":20313,"num_write":497,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:11:11.425691+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:37.399663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"178'583","reported_seq":4474,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775508+0000","last_change":"2024-09-21T23:12:59.618427+0000","last_active":"2024-09-21T23:13:18.775508+0000","last_peered":"2024-09-21T23:13:18.775508+0000","last_clean":"2024-09-21T23:13:18.775508+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:13:18.775508+0000","last_undegraded":"2024-09-21T23:13:18.775508+0000","last_fullsized":"2024-09-21T23:13:18.775508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:08:21.796460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070044759999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5321,"num_read_kb":19543,"num_write":664,"num_write_kb":14858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":371,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893997+0000","last_change":"2024-09-21T23:09:47.723063+0000","last_active":"2024-09-21T23:13:18.893997+0000","last_peered":"2024-09-21T23:13:18.893997+0000","last_clean":"2024-09-21T23:13:18.893997+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:13:18.893997+0000","last_undegraded":"2024-09-21T23:13:18.893997+0000","last_fullsized":"2024-09-21T23:13:18.893997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:44.676413+0000","last_clean_scrub_stamp":"2024-09-21T23:09:44.676413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:00.541482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":470,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.893940+0000","last_change":"2024-09-21T23:11:14.446739+0000","last_active":"2024-09-21T23:13:18.893940+0000","last_peered":"2024-09-21T23:13:18.893940+0000","last_clean":"2024-09-21T23:13:18.893940+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:13:18.893940+0000","last_undegraded":"2024-09-21T23:13:18.893940+0000","last_fullsized":"2024-09-21T23:13:18.893940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:40.631524+0000","last_clean_scrub_stamp":"2024-09-21T23:09:40.631524+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:35.422019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'543","reported_seq":3852,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775721+0000","last_change":"2024-09-21T23:12:59.615280+0000","last_active":"2024-09-21T23:13:18.775721+0000","last_peered":"2024-09-21T23:13:18.775721+0000","last_clean":"2024-09-21T23:13:18.775721+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:13:18.775721+0000","last_undegraded":"2024-09-21T23:13:18.775721+0000","last_fullsized":"2024-09-21T23:13:18.775721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:55:07.747469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040489300000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4907,"num_read_kb":10236,"num_write":527,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"174'562","reported_seq":4608,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.772591+0000","last_change":"2024-09-21T23:12:59.616991+0000","last_active":"2024-09-21T23:13:18.772591+0000","last_peered":"2024-09-21T23:13:18.772591+0000","last_clean":"2024-09-21T23:13:18.772591+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:13:18.772591+0000","last_undegraded":"2024-09-21T23:13:18.772591+0000","last_fullsized":"2024-09-21T23:13:18.772591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:16.424894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062271990000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6420,"num_read_kb":19858,"num_write":506,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"177'650","reported_seq":6686,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895221+0000","last_change":"2024-09-21T23:13:07.427624+0000","last_active":"2024-09-21T23:13:18.895221+0000","last_peered":"2024-09-21T23:13:18.895221+0000","last_clean":"2024-09-21T23:13:18.895221+0000","last_became_active":"2024-09-21T23:13:07.423010+0000","last_became_peered":"2024-09-21T23:13:07.423010+0000","last_unstale":"2024-09-21T23:13:18.895221+0000","last_undegraded":"2024-09-21T23:13:18.895221+0000","last_fullsized":"2024-09-21T23:13:18.895221+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:00.226294+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059613299999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":6967,"num_read_kb":40354,"num_write":560,"num_write_kb":11935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"174'417","reported_seq":3793,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.775569+0000","last_change":"2024-09-21T23:12:59.620627+0000","last_active":"2024-09-21T23:13:18.775569+0000","last_peered":"2024-09-21T23:13:18.775569+0000","last_clean":"2024-09-21T23:13:18.775569+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:13:18.775569+0000","last_undegraded":"2024-09-21T23:13:18.775569+0000","last_fullsized":"2024-09-21T23:13:18.775569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:19.532943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009141883,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3542,"num_read_kb":26420,"num_write":356,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'558","reported_seq":2492,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894778+0000","last_change":"2024-09-21T23:12:59.620407+0000","last_active":"2024-09-21T23:13:18.894778+0000","last_peered":"2024-09-21T23:13:18.894778+0000","last_clean":"2024-09-21T23:13:18.894778+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:13:18.894778+0000","last_undegraded":"2024-09-21T23:13:18.894778+0000","last_fullsized":"2024-09-21T23:13:18.894778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:54:14.289431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051982479999999999,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2495,"num_read_kb":17923,"num_write":507,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"175'537","reported_seq":5294,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894489+0000","last_change":"2024-09-21T23:12:59.623819+0000","last_active":"2024-09-21T23:13:18.894489+0000","last_peered":"2024-09-21T23:13:18.894489+0000","last_clean":"2024-09-21T23:13:18.894489+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:13:18.894489+0000","last_undegraded":"2024-09-21T23:13:18.894489+0000","last_fullsized":"2024-09-21T23:13:18.894489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:17.779188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004612685,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5087,"num_read_kb":28815,"num_write":438,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"174'511","reported_seq":5211,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894634+0000","last_change":"2024-09-21T23:12:59.622033+0000","last_active":"2024-09-21T23:13:18.894634+0000","last_peered":"2024-09-21T23:13:18.894634+0000","last_clean":"2024-09-21T23:13:18.894634+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:13:18.894634+0000","last_undegraded":"2024-09-21T23:13:18.894634+0000","last_fullsized":"2024-09-21T23:13:18.894634+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:26.432233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033040949999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6927,"num_read_kb":23882,"num_write":481,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"173'593","reported_seq":2677,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894638+0000","last_change":"2024-09-21T23:12:59.621649+0000","last_active":"2024-09-21T23:13:18.894638+0000","last_peered":"2024-09-21T23:13:18.894638+0000","last_clean":"2024-09-21T23:13:18.894638+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:13:18.894638+0000","last_undegraded":"2024-09-21T23:13:18.894638+0000","last_fullsized":"2024-09-21T23:13:18.894638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:34.711481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027323970000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2413,"num_read_kb":21711,"num_write":633,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'671","reported_seq":4520,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.895178+0000","last_change":"2024-09-21T23:13:06.426028+0000","last_active":"2024-09-21T23:13:18.895178+0000","last_peered":"2024-09-21T23:13:18.895178+0000","last_clean":"2024-09-21T23:13:18.895178+0000","last_became_active":"2024-09-21T23:13:06.420853+0000","last_became_peered":"2024-09-21T23:13:06.420853+0000","last_unstale":"2024-09-21T23:13:18.895178+0000","last_undegraded":"2024-09-21T23:13:18.895178+0000","last_fullsized":"2024-09-21T23:13:18.895178+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:29.389023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062394340000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6106,"num_read_kb":37597,"num_write":682,"num_write_kb":13471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"172'641","reported_seq":7006,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894240+0000","last_change":"2024-09-21T23:12:59.618210+0000","last_active":"2024-09-21T23:13:18.894240+0000","last_peered":"2024-09-21T23:13:18.894240+0000","last_clean":"2024-09-21T23:13:18.894240+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:13:18.894240+0000","last_undegraded":"2024-09-21T23:13:18.894240+0000","last_fullsized":"2024-09-21T23:13:18.894240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:43.253945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011826829999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":7995,"num_read_kb":42900,"num_write":666,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'627","reported_seq":7308,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894077+0000","last_change":"2024-09-21T23:13:05.416690+0000","last_active":"2024-09-21T23:13:18.894077+0000","last_peered":"2024-09-21T23:13:18.894077+0000","last_clean":"2024-09-21T23:13:18.894077+0000","last_became_active":"2024-09-21T23:13:05.415127+0000","last_became_peered":"2024-09-21T23:13:05.415127+0000","last_unstale":"2024-09-21T23:13:18.894077+0000","last_undegraded":"2024-09-21T23:13:18.894077+0000","last_fullsized":"2024-09-21T23:13:18.894077+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032033349999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8199,"num_read_kb":39748,"num_write":659,"num_write_kb":21750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'413","reported_seq":2638,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:18.894555+0000","last_change":"2024-09-21T23:12:59.620277+0000","last_active":"2024-09-21T23:13:18.894555+0000","last_peered":"2024-09-21T23:13:18.894555+0000","last_clean":"2024-09-21T23:13:18.894555+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:13:18.894555+0000","last_undegraded":"2024-09-21T23:13:18.894555+0000","last_fullsized":"2024-09-21T23:13:18.894555+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:09:54.792259+0000","last_clean_scrub_stamp":"2024-09-21T23:09:54.792259+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:07.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562690000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":1457,"num_read_kb":14145,"num_write":378,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":720,"num_read":186161,"num_read_kb":840774,"num_write":22415,"num_write_kb":399960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5301,"num_bytes_recovered":254407838,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":20934,"ondisk_log_size":20934,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29348,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342492,"statfs":{"total":96636764160,"available":96606711808,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743455,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43424,"kb_used_data":2316,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94328416,"statfs":{"total":96636764160,"available":96592297984,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77916,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293924,"statfs":{"total":96636764160,"available":96556978176,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29340,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342500,"statfs":{"total":96636764160,"available":96606720000,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55716,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316124,"statfs":{"total":96636764160,"available":96579710976,"internally_reserved":0,"allocated":2375680,"data_stored":1149820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:13:29.062 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.062 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.062 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.063 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.064 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T23:09:44.676413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=44, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T23:09:40.631524+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=40, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.065 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.066 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T23:09:54.792259+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=9, tm_sec=54, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=13, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:13:29.067 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T23:13:49.068 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:13:49.378 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-21T23:13:49.379 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-09-21T23:13:49.395 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-21T23:13:48.557492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":728,"num_whiteouts":720,"num_read":186277,"num_read_kb":840874,"num_write":22576,"num_write_kb":402378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5305,"num_bytes_recovered":255588574,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21781,"ondisk_log_size":21781,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394264,"kb_used_data":19672,"kb_used_omap":25,"kb_used_meta":374438,"kb_avail":754580456,"statfs":{"total":773094113280,"available":772690386944,"internally_reserved":0,"allocated":20144128,"data_stored":10379068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26577,"internal_metadata":383424559},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004181"},"pg_stats":[{"pgid":"3.1f","version":"196'606","reported_seq":4256,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:31.266928+0000","last_change":"2024-09-21T23:13:31.266992+0000","last_active":"2024-09-21T23:13:31.266928+0000","last_peered":"2024-09-21T23:13:31.266928+0000","last_clean":"2024-09-21T23:13:31.266928+0000","last_became_active":"2024-09-21T23:13:05.423231+0000","last_became_peered":"2024-09-21T23:13:05.423231+0000","last_unstale":"2024-09-21T23:13:31.266928+0000","last_undegraded":"2024-09-21T23:13:31.266928+0000","last_fullsized":"2024-09-21T23:13:31.266928+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'606","last_scrub_stamp":"2024-09-21T23:13:31.266734+0000","last_deep_scrub":"196'606","last_deep_scrub_stamp":"2024-09-21T23:13:31.266734+0000","last_clean_scrub_stamp":"2024-09-21T23:13:31.266734+0000","objects_scrubbed":26,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:33:11.589725+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0.0087587719999999997,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6355,"num_read_kb":19114,"num_write":548,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"196'915","reported_seq":4795,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:32.353549+0000","last_change":"2024-09-21T23:13:32.353738+0000","last_active":"2024-09-21T23:13:32.353549+0000","last_peered":"2024-09-21T23:13:32.353549+0000","last_clean":"2024-09-21T23:13:32.353549+0000","last_became_active":"2024-09-21T23:13:07.421837+0000","last_became_peered":"2024-09-21T23:13:07.421837+0000","last_unstale":"2024-09-21T23:13:32.353549+0000","last_undegraded":"2024-09-21T23:13:32.353549+0000","last_fullsized":"2024-09-21T23:13:32.353549+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'915","last_scrub_stamp":"2024-09-21T23:13:32.353312+0000","last_deep_scrub":"196'915","last_deep_scrub_stamp":"2024-09-21T23:13:32.353312+0000","last_clean_scrub_stamp":"2024-09-21T23:13:32.353312+0000","objects_scrubbed":21,"log_size":915,"log_dups_size":0,"ondisk_log_size":915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:20:35.767000+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.001882955,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5630,"num_read_kb":41276,"num_write":1148,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"196'477","reported_seq":3487,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:32.674924+0000","last_change":"2024-09-21T23:13:32.675029+0000","last_active":"2024-09-21T23:13:32.674924+0000","last_peered":"2024-09-21T23:13:32.674924+0000","last_clean":"2024-09-21T23:13:32.674924+0000","last_became_active":"2024-09-21T23:11:05.379947+0000","last_became_peered":"2024-09-21T23:11:05.379947+0000","last_unstale":"2024-09-21T23:13:32.674924+0000","last_undegraded":"2024-09-21T23:13:32.674924+0000","last_fullsized":"2024-09-21T23:13:32.674924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'477","last_scrub_stamp":"2024-09-21T23:13:32.674724+0000","last_deep_scrub":"196'477","last_deep_scrub_stamp":"2024-09-21T23:13:32.674724+0000","last_clean_scrub_stamp":"2024-09-21T23:13:32.674724+0000","objects_scrubbed":21,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:25.168457+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.00158018,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3362,"num_read_kb":13834,"num_write":377,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2220935,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"196'514","reported_seq":5862,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:33.651152+0000","last_change":"2024-09-21T23:13:33.651296+0000","last_active":"2024-09-21T23:13:33.651152+0000","last_peered":"2024-09-21T23:13:33.651152+0000","last_clean":"2024-09-21T23:13:33.651152+0000","last_became_active":"2024-09-21T23:11:07.385160+0000","last_became_peered":"2024-09-21T23:11:07.385160+0000","last_unstale":"2024-09-21T23:13:33.651152+0000","last_undegraded":"2024-09-21T23:13:33.651152+0000","last_fullsized":"2024-09-21T23:13:33.651152+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'514","last_scrub_stamp":"2024-09-21T23:13:33.650928+0000","last_deep_scrub":"196'514","last_deep_scrub_stamp":"2024-09-21T23:13:33.650928+0000","last_clean_scrub_stamp":"2024-09-21T23:13:33.650928+0000","objects_scrubbed":14,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:54:09.815094+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0037616780000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5378,"num_read_kb":22578,"num_write":533,"num_write_kb":12630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":2536398,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"196'641","reported_seq":4519,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:29.265790+0000","last_change":"2024-09-21T23:13:29.265880+0000","last_active":"2024-09-21T23:13:29.265790+0000","last_peered":"2024-09-21T23:13:29.265790+0000","last_clean":"2024-09-21T23:13:29.265790+0000","last_became_active":"2024-09-21T23:13:05.421427+0000","last_became_peered":"2024-09-21T23:13:05.421427+0000","last_unstale":"2024-09-21T23:13:29.265790+0000","last_undegraded":"2024-09-21T23:13:29.265790+0000","last_fullsized":"2024-09-21T23:13:29.265790+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'641","last_scrub_stamp":"2024-09-21T23:13:29.265615+0000","last_deep_scrub":"196'641","last_deep_scrub_stamp":"2024-09-21T23:13:29.265615+0000","last_clean_scrub_stamp":"2024-09-21T23:13:29.265615+0000","objects_scrubbed":23,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:02:49.893653+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.010738146,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5513,"num_read_kb":28043,"num_write":694,"num_write_kb":17938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":9587842,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"196'1351","reported_seq":7960,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:30.174750+0000","last_change":"2024-09-21T23:13:30.174819+0000","last_active":"2024-09-21T23:13:30.174750+0000","last_peered":"2024-09-21T23:13:30.174750+0000","last_clean":"2024-09-21T23:13:30.174750+0000","last_became_active":"2024-09-21T23:12:28.622726+0000","last_became_peered":"2024-09-21T23:12:28.622726+0000","last_unstale":"2024-09-21T23:13:30.174750+0000","last_undegraded":"2024-09-21T23:13:30.174750+0000","last_fullsized":"2024-09-21T23:13:30.174750+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1351","last_scrub_stamp":"2024-09-21T23:13:30.174561+0000","last_deep_scrub":"196'1351","last_deep_scrub_stamp":"2024-09-21T23:13:30.174561+0000","last_clean_scrub_stamp":"2024-09-21T23:13:30.174561+0000","objects_scrubbed":28,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:24:28.819203+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0.0046765799999999996,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":12289,"num_read_kb":29931,"num_write":1690,"num_write_kb":9650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":12429281,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"196'1034","reported_seq":3192,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:27.317392+0000","last_change":"2024-09-21T23:13:27.317415+0000","last_active":"2024-09-21T23:13:27.317392+0000","last_peered":"2024-09-21T23:13:27.317392+0000","last_clean":"2024-09-21T23:13:27.317392+0000","last_became_active":"2024-09-21T23:09:55.364686+0000","last_became_peered":"2024-09-21T23:09:55.364686+0000","last_unstale":"2024-09-21T23:13:27.317392+0000","last_undegraded":"2024-09-21T23:13:27.317392+0000","last_fullsized":"2024-09-21T23:13:27.317392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1034","last_scrub_stamp":"2024-09-21T23:13:27.317321+0000","last_deep_scrub":"196'1034","last_deep_scrub_stamp":"2024-09-21T23:13:27.317321+0000","last_clean_scrub_stamp":"2024-09-21T23:13:27.317321+0000","objects_scrubbed":25,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:13.255998+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.006449682,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":6326,"num_read_kb":21971,"num_write":1459,"num_write_kb":13876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"196'1075","reported_seq":5574,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:31.329466+0000","last_change":"2024-09-21T23:13:31.329570+0000","last_active":"2024-09-21T23:13:31.329466+0000","last_peered":"2024-09-21T23:13:31.329466+0000","last_clean":"2024-09-21T23:13:31.329466+0000","last_became_active":"2024-09-21T23:12:37.444303+0000","last_became_peered":"2024-09-21T23:12:37.444303+0000","last_unstale":"2024-09-21T23:13:31.329466+0000","last_undegraded":"2024-09-21T23:13:31.329466+0000","last_fullsized":"2024-09-21T23:13:31.329466+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1075","last_scrub_stamp":"2024-09-21T23:13:31.329263+0000","last_deep_scrub":"196'1075","last_deep_scrub_stamp":"2024-09-21T23:13:31.329263+0000","last_clean_scrub_stamp":"2024-09-21T23:13:31.329263+0000","objects_scrubbed":18,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:24.489716+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0030213219999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8120,"num_read_kb":28946,"num_write":1438,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5999502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"196'681","reported_seq":5731,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:30.267419+0000","last_change":"2024-09-21T23:13:30.267548+0000","last_active":"2024-09-21T23:13:30.267419+0000","last_peered":"2024-09-21T23:13:30.267419+0000","last_clean":"2024-09-21T23:13:30.267419+0000","last_became_active":"2024-09-21T23:13:05.423133+0000","last_became_peered":"2024-09-21T23:13:05.423133+0000","last_unstale":"2024-09-21T23:13:30.267419+0000","last_undegraded":"2024-09-21T23:13:30.267419+0000","last_fullsized":"2024-09-21T23:13:30.267419+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'681","last_scrub_stamp":"2024-09-21T23:13:30.267201+0000","last_deep_scrub":"196'681","last_deep_scrub_stamp":"2024-09-21T23:13:30.267201+0000","last_clean_scrub_stamp":"2024-09-21T23:13:30.267201+0000","objects_scrubbed":29,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:28.882585+0000","scrub_duration":51,"objects_trimmed":2,"snaptrim_duration":0.0022352600000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":6037,"num_read_kb":23064,"num_write":602,"num_write_kb":12198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"196'544","reported_seq":4617,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:34.667759+0000","last_change":"2024-09-21T23:13:34.667841+0000","last_active":"2024-09-21T23:13:34.667759+0000","last_peered":"2024-09-21T23:13:34.667759+0000","last_clean":"2024-09-21T23:13:34.667759+0000","last_became_active":"2024-09-21T23:11:14.540220+0000","last_became_peered":"2024-09-21T23:11:14.540220+0000","last_unstale":"2024-09-21T23:13:34.667759+0000","last_undegraded":"2024-09-21T23:13:34.667759+0000","last_fullsized":"2024-09-21T23:13:34.667759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'544","last_scrub_stamp":"2024-09-21T23:13:34.667651+0000","last_deep_scrub":"196'544","last_deep_scrub_stamp":"2024-09-21T23:13:34.667651+0000","last_clean_scrub_stamp":"2024-09-21T23:13:34.667651+0000","objects_scrubbed":23,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:32:16.966079+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0.0066845790000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5732,"num_read_kb":29704,"num_write":420,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":15016597,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"196'577","reported_seq":7004,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:32.654637+0000","last_change":"2024-09-21T23:13:32.654637+0000","last_active":"2024-09-21T23:13:32.654637+0000","last_peered":"2024-09-21T23:13:32.654637+0000","last_clean":"2024-09-21T23:13:32.654637+0000","last_became_active":"2024-09-21T23:11:14.446331+0000","last_became_peered":"2024-09-21T23:11:14.446331+0000","last_unstale":"2024-09-21T23:13:32.654637+0000","last_undegraded":"2024-09-21T23:13:32.654637+0000","last_fullsized":"2024-09-21T23:13:32.654637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'577","last_scrub_stamp":"2024-09-21T23:13:32.654145+0000","last_deep_scrub":"196'577","last_deep_scrub_stamp":"2024-09-21T23:13:32.654145+0000","last_clean_scrub_stamp":"2024-09-21T23:13:32.654145+0000","objects_scrubbed":16,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:55.529850+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0048388729999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7517,"num_read_kb":36655,"num_write":459,"num_write_kb":15039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":10155444,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"196'725","reported_seq":5121,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:31.162520+0000","last_change":"2024-09-21T23:13:31.162642+0000","last_active":"2024-09-21T23:13:31.162520+0000","last_peered":"2024-09-21T23:13:31.162520+0000","last_clean":"2024-09-21T23:13:31.162520+0000","last_became_active":"2024-09-21T23:12:29.629303+0000","last_became_peered":"2024-09-21T23:12:29.629303+0000","last_unstale":"2024-09-21T23:13:31.162520+0000","last_undegraded":"2024-09-21T23:13:31.162520+0000","last_fullsized":"2024-09-21T23:13:31.162520+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'725","last_scrub_stamp":"2024-09-21T23:13:31.162330+0000","last_deep_scrub":"196'725","last_deep_scrub_stamp":"2024-09-21T23:13:31.162330+0000","last_clean_scrub_stamp":"2024-09-21T23:13:31.162330+0000","objects_scrubbed":22,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:51.683910+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0024929100000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5571,"num_read_kb":14116,"num_write":826,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11596666,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:28.876751+0000","last_change":"2024-09-21T23:13:28.876830+0000","last_active":"2024-09-21T23:13:28.876751+0000","last_peered":"2024-09-21T23:13:28.876751+0000","last_clean":"2024-09-21T23:13:28.876751+0000","last_became_active":"2024-09-21T23:13:03.806956+0000","last_became_peered":"2024-09-21T23:13:03.806956+0000","last_unstale":"2024-09-21T23:13:28.876751+0000","last_undegraded":"2024-09-21T23:13:28.876751+0000","last_fullsized":"2024-09-21T23:13:28.876751+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:13:28.876552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:13:28.876552+0000","last_clean_scrub_stamp":"2024-09-21T23:13:28.876552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:06:37.390485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"196'640","reported_seq":2828,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:26.208099+0000","last_change":"2024-09-21T23:13:26.208299+0000","last_active":"2024-09-21T23:13:26.208099+0000","last_peered":"2024-09-21T23:13:26.208099+0000","last_clean":"2024-09-21T23:13:26.208099+0000","last_became_active":"2024-09-21T23:13:05.415035+0000","last_became_peered":"2024-09-21T23:13:05.415035+0000","last_unstale":"2024-09-21T23:13:26.208099+0000","last_undegraded":"2024-09-21T23:13:26.208099+0000","last_fullsized":"2024-09-21T23:13:26.208099+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'640","last_scrub_stamp":"2024-09-21T23:13:26.207893+0000","last_deep_scrub":"196'640","last_deep_scrub_stamp":"2024-09-21T23:13:26.207893+0000","last_clean_scrub_stamp":"2024-09-21T23:13:26.207893+0000","objects_scrubbed":23,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:38.302319+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0043940419999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3092,"num_read_kb":25059,"num_write":602,"num_write_kb":13129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":388,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:30.244413+0000","last_change":"2024-09-21T23:13:30.244464+0000","last_active":"2024-09-21T23:13:30.244413+0000","last_peered":"2024-09-21T23:13:30.244413+0000","last_clean":"2024-09-21T23:13:30.244413+0000","last_became_active":"2024-09-21T23:13:03.643364+0000","last_became_peered":"2024-09-21T23:13:03.643364+0000","last_unstale":"2024-09-21T23:13:30.244413+0000","last_undegraded":"2024-09-21T23:13:30.244413+0000","last_fullsized":"2024-09-21T23:13:30.244413+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:13:30.244212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:13:30.244212+0000","last_clean_scrub_stamp":"2024-09-21T23:13:30.244212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:09:59.402562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"196'790","reported_seq":6005,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:30.297036+0000","last_change":"2024-09-21T23:13:30.297146+0000","last_active":"2024-09-21T23:13:30.297036+0000","last_peered":"2024-09-21T23:13:30.297036+0000","last_clean":"2024-09-21T23:13:30.297036+0000","last_became_active":"2024-09-21T23:11:15.381795+0000","last_became_peered":"2024-09-21T23:11:15.381795+0000","last_unstale":"2024-09-21T23:13:30.297036+0000","last_undegraded":"2024-09-21T23:13:30.297036+0000","last_fullsized":"2024-09-21T23:13:30.297036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'790","last_scrub_stamp":"2024-09-21T23:13:30.296859+0000","last_deep_scrub":"196'790","last_deep_scrub_stamp":"2024-09-21T23:13:30.296859+0000","last_clean_scrub_stamp":"2024-09-21T23:13:30.296859+0000","objects_scrubbed":17,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:24.491468+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0086643399999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8038,"num_read_kb":30164,"num_write":1035,"num_write_kb":19449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":23233531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":253,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:30.617076+0000","last_change":"2024-09-21T23:13:30.617133+0000","last_active":"2024-09-21T23:13:30.617076+0000","last_peered":"2024-09-21T23:13:30.617076+0000","last_clean":"2024-09-21T23:13:30.617076+0000","last_became_active":"2024-09-21T23:11:11.425691+0000","last_became_peered":"2024-09-21T23:11:11.425691+0000","last_unstale":"2024-09-21T23:13:30.617076+0000","last_undegraded":"2024-09-21T23:13:30.617076+0000","last_fullsized":"2024-09-21T23:13:30.617076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:13:30.616883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:13:30.616883+0000","last_clean_scrub_stamp":"2024-09-21T23:13:30.616883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:31:21.128984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"196'688","reported_seq":4048,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:29.331338+0000","last_change":"2024-09-21T23:13:29.331449+0000","last_active":"2024-09-21T23:13:29.331338+0000","last_peered":"2024-09-21T23:13:29.331338+0000","last_clean":"2024-09-21T23:13:29.331338+0000","last_became_active":"2024-09-21T23:09:55.766011+0000","last_became_peered":"2024-09-21T23:09:55.766011+0000","last_unstale":"2024-09-21T23:13:29.331338+0000","last_undegraded":"2024-09-21T23:13:29.331338+0000","last_fullsized":"2024-09-21T23:13:29.331338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'688","last_scrub_stamp":"2024-09-21T23:13:29.331171+0000","last_deep_scrub":"196'688","last_deep_scrub_stamp":"2024-09-21T23:13:29.331171+0000","last_clean_scrub_stamp":"2024-09-21T23:13:29.331171+0000","objects_scrubbed":21,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:22.124165+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0043881010000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5200,"num_read_kb":24165,"num_write":600,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":378,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:26.259766+0000","last_change":"2024-09-21T23:13:26.259788+0000","last_active":"2024-09-21T23:13:26.259766+0000","last_peered":"2024-09-21T23:13:26.259766+0000","last_clean":"2024-09-21T23:13:26.259766+0000","last_became_active":"2024-09-21T23:09:45.866688+0000","last_became_peered":"2024-09-21T23:09:45.866688+0000","last_unstale":"2024-09-21T23:13:26.259766+0000","last_undegraded":"2024-09-21T23:13:26.259766+0000","last_fullsized":"2024-09-21T23:13:26.259766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:13:26.259686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:13:26.259686+0000","last_clean_scrub_stamp":"2024-09-21T23:13:26.259686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:37:07.471684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"196'850","reported_seq":6113,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:30.609385+0000","last_change":"2024-09-21T23:13:30.609488+0000","last_active":"2024-09-21T23:13:30.609385+0000","last_peered":"2024-09-21T23:13:30.609385+0000","last_clean":"2024-09-21T23:13:30.609385+0000","last_became_active":"2024-09-21T23:09:55.381971+0000","last_became_peered":"2024-09-21T23:09:55.381971+0000","last_unstale":"2024-09-21T23:13:30.609385+0000","last_undegraded":"2024-09-21T23:13:30.609385+0000","last_fullsized":"2024-09-21T23:13:30.609385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'850","last_scrub_stamp":"2024-09-21T23:13:30.609190+0000","last_deep_scrub":"196'850","last_deep_scrub_stamp":"2024-09-21T23:13:30.609190+0000","last_clean_scrub_stamp":"2024-09-21T23:13:30.609190+0000","objects_scrubbed":21,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:14:20.732887+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0079884020000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6855,"num_read_kb":41392,"num_write":1124,"num_write_kb":18322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"196'853","reported_seq":4674,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:28.203601+0000","last_change":"2024-09-21T23:13:28.203690+0000","last_active":"2024-09-21T23:13:28.203601+0000","last_peered":"2024-09-21T23:13:28.203601+0000","last_clean":"2024-09-21T23:13:28.203601+0000","last_became_active":"2024-09-21T23:11:07.386900+0000","last_became_peered":"2024-09-21T23:11:07.386900+0000","last_unstale":"2024-09-21T23:13:28.203601+0000","last_undegraded":"2024-09-21T23:13:28.203601+0000","last_fullsized":"2024-09-21T23:13:28.203601+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'853","last_scrub_stamp":"2024-09-21T23:13:28.203414+0000","last_deep_scrub":"196'853","last_deep_scrub_stamp":"2024-09-21T23:13:28.203414+0000","last_clean_scrub_stamp":"2024-09-21T23:13:28.203414+0000","objects_scrubbed":21,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:48.774359+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.014050681000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":5872,"num_read_kb":26777,"num_write":811,"num_write_kb":12147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":3431068,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"196'3","reported_seq":382,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:28.570424+0000","last_change":"2024-09-21T23:13:28.570574+0000","last_active":"2024-09-21T23:13:28.570424+0000","last_peered":"2024-09-21T23:13:28.570424+0000","last_clean":"2024-09-21T23:13:28.570424+0000","last_became_active":"2024-09-21T23:09:45.714507+0000","last_became_peered":"2024-09-21T23:09:45.714507+0000","last_unstale":"2024-09-21T23:13:28.570424+0000","last_undegraded":"2024-09-21T23:13:28.570424+0000","last_fullsized":"2024-09-21T23:13:28.570424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'3","last_scrub_stamp":"2024-09-21T23:13:28.570212+0000","last_deep_scrub":"196'3","last_deep_scrub_stamp":"2024-09-21T23:13:28.570212+0000","last_clean_scrub_stamp":"2024-09-21T23:13:28.570212+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:42:35.534180+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0020827010000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"196'597","reported_seq":4753,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:28.274651+0000","last_change":"2024-09-21T23:13:28.274709+0000","last_active":"2024-09-21T23:13:28.274651+0000","last_peered":"2024-09-21T23:13:28.274651+0000","last_clean":"2024-09-21T23:13:28.274651+0000","last_became_active":"2024-09-21T23:09:55.765731+0000","last_became_peered":"2024-09-21T23:09:55.765731+0000","last_unstale":"2024-09-21T23:13:28.274651+0000","last_undegraded":"2024-09-21T23:13:28.274651+0000","last_fullsized":"2024-09-21T23:13:28.274651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'597","last_scrub_stamp":"2024-09-21T23:13:28.274514+0000","last_deep_scrub":"196'597","last_deep_scrub_stamp":"2024-09-21T23:13:28.274514+0000","last_clean_scrub_stamp":"2024-09-21T23:13:28.274514+0000","objects_scrubbed":26,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:56:27.272691+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0061931130000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6676,"num_read_kb":20540,"num_write":495,"num_write_kb":13141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":378,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:26.465218+0000","last_change":"2024-09-21T23:13:26.465275+0000","last_active":"2024-09-21T23:13:26.465218+0000","last_peered":"2024-09-21T23:13:26.465218+0000","last_clean":"2024-09-21T23:13:26.465218+0000","last_became_active":"2024-09-21T23:09:45.709174+0000","last_became_peered":"2024-09-21T23:09:45.709174+0000","last_unstale":"2024-09-21T23:13:26.465218+0000","last_undegraded":"2024-09-21T23:13:26.465218+0000","last_fullsized":"2024-09-21T23:13:26.465218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:13:26.464976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:13:26.464976+0000","last_clean_scrub_stamp":"2024-09-21T23:13:26.464976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:36.406899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112961,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"196'487","reported_seq":5734,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:28.277297+0000","last_change":"2024-09-21T23:13:28.277369+0000","last_active":"2024-09-21T23:13:28.277297+0000","last_peered":"2024-09-21T23:13:28.277297+0000","last_clean":"2024-09-21T23:13:28.277297+0000","last_became_active":"2024-09-21T23:13:05.421575+0000","last_became_peered":"2024-09-21T23:13:05.421575+0000","last_unstale":"2024-09-21T23:13:28.277297+0000","last_undegraded":"2024-09-21T23:13:28.277297+0000","last_fullsized":"2024-09-21T23:13:28.277297+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'487","last_scrub_stamp":"2024-09-21T23:13:28.277222+0000","last_deep_scrub":"196'487","last_deep_scrub_stamp":"2024-09-21T23:13:28.277222+0000","last_clean_scrub_stamp":"2024-09-21T23:13:28.277222+0000","objects_scrubbed":23,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:30:22.489444+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.013442716,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4762,"num_read_kb":20313,"num_write":497,"num_write_kb":9790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":10677066,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":253,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:29.580905+0000","last_change":"2024-09-21T23:13:29.580963+0000","last_active":"2024-09-21T23:13:29.580905+0000","last_peered":"2024-09-21T23:13:29.580905+0000","last_clean":"2024-09-21T23:13:29.580905+0000","last_became_active":"2024-09-21T23:11:11.424842+0000","last_became_peered":"2024-09-21T23:11:11.424842+0000","last_unstale":"2024-09-21T23:13:29.580905+0000","last_undegraded":"2024-09-21T23:13:29.580905+0000","last_fullsized":"2024-09-21T23:13:29.580905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:13:29.580662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:13:29.580662+0000","last_clean_scrub_stamp":"2024-09-21T23:13:29.580662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:39.380636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"196'602","reported_seq":4502,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:29.321265+0000","last_change":"2024-09-21T23:13:29.321393+0000","last_active":"2024-09-21T23:13:29.321265+0000","last_peered":"2024-09-21T23:13:29.321265+0000","last_clean":"2024-09-21T23:13:29.321265+0000","last_became_active":"2024-09-21T23:09:55.371388+0000","last_became_peered":"2024-09-21T23:09:55.371388+0000","last_unstale":"2024-09-21T23:13:29.321265+0000","last_undegraded":"2024-09-21T23:13:29.321265+0000","last_fullsized":"2024-09-21T23:13:29.321265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'602","last_scrub_stamp":"2024-09-21T23:13:29.320968+0000","last_deep_scrub":"196'602","last_deep_scrub_stamp":"2024-09-21T23:13:29.320968+0000","last_clean_scrub_stamp":"2024-09-21T23:13:29.320968+0000","objects_scrubbed":19,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:35:12.174967+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0070044759999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5321,"num_read_kb":19543,"num_write":664,"num_write_kb":14858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"196'2","reported_seq":381,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:29.571642+0000","last_change":"2024-09-21T23:13:29.571779+0000","last_active":"2024-09-21T23:13:29.571642+0000","last_peered":"2024-09-21T23:13:29.571642+0000","last_clean":"2024-09-21T23:13:29.571642+0000","last_became_active":"2024-09-21T23:09:45.715221+0000","last_became_peered":"2024-09-21T23:09:45.715221+0000","last_unstale":"2024-09-21T23:13:29.571642+0000","last_undegraded":"2024-09-21T23:13:29.571642+0000","last_fullsized":"2024-09-21T23:13:29.571642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'2","last_scrub_stamp":"2024-09-21T23:13:29.571423+0000","last_deep_scrub":"196'2","last_deep_scrub_stamp":"2024-09-21T23:13:29.571423+0000","last_clean_scrub_stamp":"2024-09-21T23:13:29.571423+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:50.228228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00093791600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"196'119","reported_seq":481,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:31.602738+0000","last_change":"2024-09-21T23:13:31.602815+0000","last_active":"2024-09-21T23:13:31.602738+0000","last_peered":"2024-09-21T23:13:31.602738+0000","last_clean":"2024-09-21T23:13:31.602738+0000","last_became_active":"2024-09-21T23:11:14.446276+0000","last_became_peered":"2024-09-21T23:11:14.446276+0000","last_unstale":"2024-09-21T23:13:31.602738+0000","last_undegraded":"2024-09-21T23:13:31.602738+0000","last_fullsized":"2024-09-21T23:13:31.602738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'119","last_scrub_stamp":"2024-09-21T23:13:31.602544+0000","last_deep_scrub":"196'119","last_deep_scrub_stamp":"2024-09-21T23:13:31.602544+0000","last_clean_scrub_stamp":"2024-09-21T23:13:31.602544+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:41.840141+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"196'563","reported_seq":3881,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:27.344091+0000","last_change":"2024-09-21T23:13:27.344151+0000","last_active":"2024-09-21T23:13:27.344091+0000","last_peered":"2024-09-21T23:13:27.344091+0000","last_clean":"2024-09-21T23:13:27.344091+0000","last_became_active":"2024-09-21T23:09:55.367469+0000","last_became_peered":"2024-09-21T23:09:55.367469+0000","last_unstale":"2024-09-21T23:13:27.344091+0000","last_undegraded":"2024-09-21T23:13:27.344091+0000","last_fullsized":"2024-09-21T23:13:27.344091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'563","last_scrub_stamp":"2024-09-21T23:13:27.344017+0000","last_deep_scrub":"196'563","last_deep_scrub_stamp":"2024-09-21T23:13:27.344017+0000","last_clean_scrub_stamp":"2024-09-21T23:13:27.344017+0000","objects_scrubbed":20,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:48:08.493581+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0040489300000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":4907,"num_read_kb":10236,"num_write":527,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"196'583","reported_seq":4638,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:29.166710+0000","last_change":"2024-09-21T23:13:29.166808+0000","last_active":"2024-09-21T23:13:29.166710+0000","last_peered":"2024-09-21T23:13:29.166710+0000","last_clean":"2024-09-21T23:13:29.166710+0000","last_became_active":"2024-09-21T23:12:27.619194+0000","last_became_peered":"2024-09-21T23:12:27.619194+0000","last_unstale":"2024-09-21T23:13:29.166710+0000","last_undegraded":"2024-09-21T23:13:29.166710+0000","last_fullsized":"2024-09-21T23:13:29.166710+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'583","last_scrub_stamp":"2024-09-21T23:13:29.166509+0000","last_deep_scrub":"196'583","last_deep_scrub_stamp":"2024-09-21T23:13:29.166509+0000","last_clean_scrub_stamp":"2024-09-21T23:13:29.166509+0000","objects_scrubbed":21,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:38:31.287292+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0062271990000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6420,"num_read_kb":19858,"num_write":506,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":6105444,"num_keys_recovered":108,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"196'676","reported_seq":6722,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:30.870850+0000","last_change":"2024-09-21T23:13:30.870998+0000","last_active":"2024-09-21T23:13:30.870850+0000","last_peered":"2024-09-21T23:13:30.870850+0000","last_clean":"2024-09-21T23:13:30.870850+0000","last_became_active":"2024-09-21T23:13:07.423010+0000","last_became_peered":"2024-09-21T23:13:07.423010+0000","last_unstale":"2024-09-21T23:13:30.870850+0000","last_undegraded":"2024-09-21T23:13:30.870850+0000","last_fullsized":"2024-09-21T23:13:30.870850+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'676","last_scrub_stamp":"2024-09-21T23:13:30.870603+0000","last_deep_scrub":"196'676","last_deep_scrub_stamp":"2024-09-21T23:13:30.870603+0000","last_clean_scrub_stamp":"2024-09-21T23:13:30.870603+0000","objects_scrubbed":26,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:00:56.510135+0000","scrub_duration":51,"objects_trimmed":2,"snaptrim_duration":0.0059613299999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":6967,"num_read_kb":40354,"num_write":560,"num_write_kb":11935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":22111796,"num_keys_recovered":128,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"196'436","reported_seq":3821,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:28.320194+0000","last_change":"2024-09-21T23:13:28.320262+0000","last_active":"2024-09-21T23:13:28.320194+0000","last_peered":"2024-09-21T23:13:28.320194+0000","last_clean":"2024-09-21T23:13:28.320194+0000","last_became_active":"2024-09-21T23:09:55.370524+0000","last_became_peered":"2024-09-21T23:09:55.370524+0000","last_unstale":"2024-09-21T23:13:28.320194+0000","last_undegraded":"2024-09-21T23:13:28.320194+0000","last_fullsized":"2024-09-21T23:13:28.320194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'436","last_scrub_stamp":"2024-09-21T23:13:28.320111+0000","last_deep_scrub":"196'436","last_deep_scrub_stamp":"2024-09-21T23:13:28.320111+0000","last_clean_scrub_stamp":"2024-09-21T23:13:28.320111+0000","objects_scrubbed":19,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:23.627742+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.009141883,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3542,"num_read_kb":26420,"num_write":356,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"196'587","reported_seq":2531,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:31.668498+0000","last_change":"2024-09-21T23:13:31.668643+0000","last_active":"2024-09-21T23:13:31.668498+0000","last_peered":"2024-09-21T23:13:31.668498+0000","last_clean":"2024-09-21T23:13:31.668498+0000","last_became_active":"2024-09-21T23:09:55.380547+0000","last_became_peered":"2024-09-21T23:09:55.380547+0000","last_unstale":"2024-09-21T23:13:31.668498+0000","last_undegraded":"2024-09-21T23:13:31.668498+0000","last_fullsized":"2024-09-21T23:13:31.668498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'587","last_scrub_stamp":"2024-09-21T23:13:31.668280+0000","last_deep_scrub":"196'587","last_deep_scrub_stamp":"2024-09-21T23:13:31.668280+0000","last_clean_scrub_stamp":"2024-09-21T23:13:31.668280+0000","objects_scrubbed":29,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:07.281201+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0051982479999999999,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2495,"num_read_kb":17923,"num_write":507,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"196'566","reported_seq":5333,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:33.660922+0000","last_change":"2024-09-21T23:13:33.661023+0000","last_active":"2024-09-21T23:13:33.660922+0000","last_peered":"2024-09-21T23:13:33.660922+0000","last_clean":"2024-09-21T23:13:33.660922+0000","last_became_active":"2024-09-21T23:11:15.388244+0000","last_became_peered":"2024-09-21T23:11:15.388244+0000","last_unstale":"2024-09-21T23:13:33.660922+0000","last_undegraded":"2024-09-21T23:13:33.660922+0000","last_fullsized":"2024-09-21T23:13:33.660922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'566","last_scrub_stamp":"2024-09-21T23:13:33.660689+0000","last_deep_scrub":"196'566","last_deep_scrub_stamp":"2024-09-21T23:13:33.660689+0000","last_clean_scrub_stamp":"2024-09-21T23:13:33.660689+0000","objects_scrubbed":29,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:05.587862+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0.004612685,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5087,"num_read_kb":28815,"num_write":438,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":19375445,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"196'535","reported_seq":5244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:37.704484+0000","last_change":"2024-09-21T23:13:37.704656+0000","last_active":"2024-09-21T23:13:37.704484+0000","last_peered":"2024-09-21T23:13:37.704484+0000","last_clean":"2024-09-21T23:13:37.704484+0000","last_became_active":"2024-09-21T23:12:28.625877+0000","last_became_peered":"2024-09-21T23:12:28.625877+0000","last_unstale":"2024-09-21T23:13:37.704484+0000","last_undegraded":"2024-09-21T23:13:37.704484+0000","last_fullsized":"2024-09-21T23:13:37.704484+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'535","last_scrub_stamp":"2024-09-21T23:13:37.704207+0000","last_deep_scrub":"196'535","last_deep_scrub_stamp":"2024-09-21T23:13:37.704207+0000","last_clean_scrub_stamp":"2024-09-21T23:13:37.704207+0000","objects_scrubbed":24,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:54.216708+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0033040949999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6927,"num_read_kb":23882,"num_write":481,"num_write_kb":9496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19903414,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"196'622","reported_seq":2716,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:34.665657+0000","last_change":"2024-09-21T23:13:34.665731+0000","last_active":"2024-09-21T23:13:34.665657+0000","last_peered":"2024-09-21T23:13:34.665657+0000","last_clean":"2024-09-21T23:13:34.665657+0000","last_became_active":"2024-09-21T23:11:16.383479+0000","last_became_peered":"2024-09-21T23:11:16.383479+0000","last_unstale":"2024-09-21T23:13:34.665657+0000","last_undegraded":"2024-09-21T23:13:34.665657+0000","last_fullsized":"2024-09-21T23:13:34.665657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'622","last_scrub_stamp":"2024-09-21T23:13:34.665454+0000","last_deep_scrub":"196'622","last_deep_scrub_stamp":"2024-09-21T23:13:34.665454+0000","last_clean_scrub_stamp":"2024-09-21T23:13:34.665454+0000","objects_scrubbed":29,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:47.040642+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0.0027323970000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":2413,"num_read_kb":21711,"num_write":633,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":12274929,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"196'692","reported_seq":4550,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:29.879597+0000","last_change":"2024-09-21T23:13:29.879690+0000","last_active":"2024-09-21T23:13:29.879597+0000","last_peered":"2024-09-21T23:13:29.879597+0000","last_clean":"2024-09-21T23:13:29.879597+0000","last_became_active":"2024-09-21T23:13:06.420853+0000","last_became_peered":"2024-09-21T23:13:06.420853+0000","last_unstale":"2024-09-21T23:13:29.879597+0000","last_undegraded":"2024-09-21T23:13:29.879597+0000","last_fullsized":"2024-09-21T23:13:29.879597+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'692","last_scrub_stamp":"2024-09-21T23:13:29.879386+0000","last_deep_scrub":"196'692","last_deep_scrub_stamp":"2024-09-21T23:13:29.879386+0000","last_clean_scrub_stamp":"2024-09-21T23:13:29.879386+0000","objects_scrubbed":21,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:24.507639+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0062394340000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6106,"num_read_kb":37597,"num_write":682,"num_write_kb":13471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"196'669","reported_seq":7045,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:35.671939+0000","last_change":"2024-09-21T23:13:35.672097+0000","last_active":"2024-09-21T23:13:35.671939+0000","last_peered":"2024-09-21T23:13:35.671939+0000","last_clean":"2024-09-21T23:13:35.671939+0000","last_became_active":"2024-09-21T23:11:16.383990+0000","last_became_peered":"2024-09-21T23:11:16.383990+0000","last_unstale":"2024-09-21T23:13:35.671939+0000","last_undegraded":"2024-09-21T23:13:35.671939+0000","last_fullsized":"2024-09-21T23:13:35.671939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'669","last_scrub_stamp":"2024-09-21T23:13:35.671725+0000","last_deep_scrub":"196'669","last_deep_scrub_stamp":"2024-09-21T23:13:35.671725+0000","last_clean_scrub_stamp":"2024-09-21T23:13:35.671725+0000","objects_scrubbed":28,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:44:08.736904+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.0011826829999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":7995,"num_read_kb":42900,"num_write":666,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":18261810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"196'646","reported_seq":7335,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:27.233318+0000","last_change":"2024-09-21T23:13:27.233415+0000","last_active":"2024-09-21T23:13:27.233318+0000","last_peered":"2024-09-21T23:13:27.233318+0000","last_clean":"2024-09-21T23:13:27.233318+0000","last_became_active":"2024-09-21T23:13:05.415127+0000","last_became_peered":"2024-09-21T23:13:05.415127+0000","last_unstale":"2024-09-21T23:13:27.233318+0000","last_undegraded":"2024-09-21T23:13:27.233318+0000","last_fullsized":"2024-09-21T23:13:27.233318+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'646","last_scrub_stamp":"2024-09-21T23:13:27.233140+0000","last_deep_scrub":"196'646","last_deep_scrub_stamp":"2024-09-21T23:13:27.233140+0000","last_clean_scrub_stamp":"2024-09-21T23:13:27.233140+0000","objects_scrubbed":19,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:39:22.157541+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0032033349999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8199,"num_read_kb":39748,"num_write":659,"num_write_kb":21750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":27248315,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"196'435","reported_seq":2670,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-21T23:13:36.702891+0000","last_change":"2024-09-21T23:13:36.702995+0000","last_active":"2024-09-21T23:13:36.702891+0000","last_peered":"2024-09-21T23:13:36.702891+0000","last_clean":"2024-09-21T23:13:36.702891+0000","last_became_active":"2024-09-21T23:12:27.617292+0000","last_became_peered":"2024-09-21T23:12:27.617292+0000","last_unstale":"2024-09-21T23:13:36.702891+0000","last_undegraded":"2024-09-21T23:13:36.702891+0000","last_fullsized":"2024-09-21T23:13:36.702891+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'435","last_scrub_stamp":"2024-09-21T23:13:36.702701+0000","last_deep_scrub":"196'435","last_deep_scrub_stamp":"2024-09-21T23:13:36.702701+0000","last_clean_scrub_stamp":"2024-09-21T23:13:36.702701+0000","objects_scrubbed":22,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:45.759025+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0021562690000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":1457,"num_read_kb":14145,"num_write":378,"num_write_kb":7693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":22242355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":720,"num_read":186161,"num_read_kb":840774,"num_write":22415,"num_write_kb":399960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5301,"num_bytes_recovered":254407838,"num_keys_recovered":713,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},"log_size":21657,"ondisk_log_size":21657,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29336,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342504,"statfs":{"total":96636764160,"available":96606724096,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37660,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":69,"seq":296352743460,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43420,"kb_used_data":2316,"kb_used_omap":15,"kb_used_meta":41072,"kb_avail":94328420,"statfs":{"total":96636764160,"available":96592302080,"internally_reserved":0,"allocated":2371584,"data_stored":1149801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15422,"internal_metadata":42058690},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77912,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293928,"statfs":{"total":96636764160,"available":96556982272,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29336,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342504,"statfs":{"total":96636764160,"available":96606724096,"internally_reserved":0,"allocated":2367488,"data_stored":1149782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55712,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316128,"statfs":{"total":96636764160,"available":96579715072,"internally_reserved":0,"allocated":2375680,"data_stored":1149820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":2961408,"data_stored":1740150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8343,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:13:49.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T23:13:49.728 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T23:13:49.728 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T23:13:49.729 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T23:13:49.729 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:13:49.820 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T23:13:49.820 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T23:13:49.820 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:13:55.922 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T23:13:55.922 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T23:13:55.922 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:13:55.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-21T23:13:55.994 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T23:13:55.994 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T23:13:55.994 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:01.852 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-21T23:14:01.853 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-21T23:14:02.095 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T23:14:02.096 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T23:14:02.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:02.159 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T23:14:02.159 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T23:14:02.160 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:02.244 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T23:14:02.244 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T23:14:02.244 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:02.332 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T23:14:02.332 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T23:14:02.333 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:07.358 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-21T23:14:07.358 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-21T23:14:08.434 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T23:14:08.434 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T23:14:08.435 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T23:14:08.435 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:08.463 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T23:14:08.463 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T23:14:08.463 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:08.494 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T23:14:08.494 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T23:14:08.494 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T23:14:08.494 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:08.528 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T23:14:08.529 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T23:14:08.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:08.585 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T23:14:08.586 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T23:14:08.586 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:14:08.606 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T23:14:08.606 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T23:14:08.607 DEBUG:teuthology.orchestra.run.smithi150:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-21T23:14:08.645 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:14:08.645 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T23:14:09.073 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:14:09.074 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T23:14:09.270 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:14:09.270 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T23:14:09.443 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:14:09.443 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T23:14:09.621 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:14:09.621 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T23:14:09.791 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:14:09.792 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T23:14:09.964 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:14:09.964 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T23:14:10.131 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:14:10.131 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T23:14:10.274 INFO:tasks.ceph:Archiving mon data... 2024-09-21T23:14:10.275 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/data/mon.a.tgz 2024-09-21T23:14:10.275 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-21T23:14:10.281 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.IiqniNxcET 2024-09-21T23:14:10.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IiqniNxcET 2024-09-21T23:14:10.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.IiqniNxcET 2024-09-21T23:14:10.505 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.IiqniNxcET is 446KB 2024-09-21T23:14:10.567 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.IiqniNxcET 2024-09-21T23:14:10.572 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/data/mon.c.tgz 2024-09-21T23:14:10.572 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-21T23:14:10.621 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.Z1j4aQO46q 2024-09-21T23:14:10.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Z1j4aQO46q 2024-09-21T23:14:10.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.Z1j4aQO46q 2024-09-21T23:14:10.872 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.Z1j4aQO46q is 613KB 2024-09-21T23:14:10.893 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.Z1j4aQO46q 2024-09-21T23:14:10.898 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/data/mon.b.tgz 2024-09-21T23:14:10.898 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-21T23:14:10.904 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.jGZNyEe0jC 2024-09-21T23:14:10.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jGZNyEe0jC 2024-09-21T23:14:11.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.jGZNyEe0jC 2024-09-21T23:14:11.156 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.jGZNyEe0jC is 613KB 2024-09-21T23:14:11.218 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.jGZNyEe0jC 2024-09-21T23:14:11.225 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T23:14:11.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:14:11.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:14:11.285 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:14:11.329 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:14:11.329 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T23:14:11.331 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/remote/smithi150/crash 2024-09-21T23:14:11.332 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:14:11.345 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/remote/smithi175/crash 2024-09-21T23:14:11.346 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:14:11.384 INFO:tasks.ceph:Compressing logs... 2024-09-21T23:14:11.385 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:14:11.390 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:14:11.413 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35894.log 2024-09-21T23:14:11.415 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32848.log 2024-09-21T23:14:11.415 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38650.log 2024-09-21T23:14:11.415 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35894.log.gz 2024-09-21T23:14:11.415 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-09-21T23:14:11.415 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32848.log.gz 2024-09-21T23:14:11.415 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38650.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38650.log.gz -- 2024-09-21T23:14:11.415 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/ceph-client.admin.41507.log 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42275.log 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41507.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41507.log.gz 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40513.log 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42275.log.gz 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40227.log 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40513.log.gz 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36922.log 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40227.log.gz 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2024-09-21T23:14:11.445 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42972.log 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42975.log 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36922.log.gz 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42972.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42972.log.gz 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40466.log 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36658.log 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42975.log.gz 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35924.log 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40466.log.gz 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39801.log 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36658.log.gz 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35924.log.gz 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43805.log 2024-09-21T23:14:11.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39801.log.gz 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43805.log.gz 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40800.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40800.log.gz 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42766.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34120.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42766.log.gz 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41313.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43389.log 2024-09-21T23:14:11.447 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34120.log.gz 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41313.log.gz 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41036.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/ceph-client.admin.43389.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36605.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43389.log.gz 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41036.log.gz 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36401.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36605.log.gz 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35551.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34362.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36401.log.gz 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35551.log.gz 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43727.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2024-09-21T23:14:11.448 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34362.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43727.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40748.log 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43753.log 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40748.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33292.log 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43753.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38683.log 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33292.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32630.log 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41424.log 2024-09-21T23:14:11.449 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38683.log.gz 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44012.log 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32630.log.gz 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32590.log 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41424.log.gz 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.44012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44012.log.gz 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43415.log 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38765.log 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32590.log.gz 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43623.log 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43415.log.gz 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38765.log.gzgzip 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42947.log 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43961.log 2024-09-21T23:14:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43623.log.gz 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33766.log 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42947.log.gz 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43961.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43961.log.gz -5 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41859.log 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33459.log 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33766.log.gz 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41859.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.41859.log.gz 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/ceph-client.admin.40083.log 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.0.33459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42323.log 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40083.log.gz 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.0.33459.log.gz 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2024-09-21T23:14:11.451 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42323.log: -- /var/log/ceph/ceph-client.admin.42083.log 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42323.log.gz 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39281.log 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42083.log.gzgzip 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39281.log.gz 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35973.log 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34479.log 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35973.log: /var/log/ceph/ceph-client.admin.33043.log 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35973.log.gz 2024-09-21T23:14:11.452 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40832.log 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34479.log.gz 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33043.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33043.log.gz 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36345.log 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40832.log: -- /var/log/ceph/ceph-client.admin.40387.log 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40832.log.gz 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37700.log 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36345.log.gz 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40387.log.gz 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38199.log 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37022.log 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37700.log.gz 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34955.log 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38199.log.gz 2024-09-21T23:14:11.453 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37022.log.gz 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32847.log 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36793.log 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34955.log.gz 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32847.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32847.log.gz 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36793.log.gz 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42531.log 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.43181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42531.log.gz 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42162.log 2024-09-21T23:14:11.454 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42162.log.gz 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33472.log 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32851.log 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37802.log 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33472.log.gz 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32851.log.gz 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31732.log 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33855.log 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37802.log.gz 2024-09-21T23:14:11.455 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.31732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31732.log.gz 2024-09-21T23:14:11.456 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35108.log 2024-09-21T23:14:11.456 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40420.log 2024-09-21T23:14:11.456 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33855.log.gz 2024-09-21T23:14:11.456 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35108.log.gz 2024-09-21T23:14:11.456 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39463.log 2024-09-21T23:14:11.456 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40420.log.gz 2024-09-21T23:14:11.456 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39557.log 2024-09-21T23:14:11.468 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26392.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26392.log.gz 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33125.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/objectstore_tool.33125.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32346.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26567.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/objectstore_tool.32346.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26440.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26567.log.gz/var/log/ceph/ceph-mgr.x.log: 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26464.log 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26440.log: /var/log/ceph/ceph.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26440.log.gz 2024-09-21T23:14:11.469 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26488.log 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26464.log.gz 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26488.log.gz 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26416.log 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26536.log 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26416.log.gz 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26512.log 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.33125.log.gz 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26536.log.gz 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.26512.log: 0.0%gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33888.log 2024-09-21T23:14:11.470 INFO:teuthology.orchestra.run.smithi175.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26512.log.gz 2024-09-21T23:14:11.471 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T23:14:11.471 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/objectstore_tool.33888.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34677.log 2024-09-21T23:14:11.471 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.4.log: 95.3% -- replaced with /var/log/ceph/objectstore_tool.32346.log.gz 2024-09-21T23:14:11.482 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/objectstore_tool.34677.log: 95.4% -- replaced with /var/log/ceph/objectstore_tool.33888.log.gz 2024-09-21T23:14:11.482 INFO:teuthology.orchestra.run.smithi175.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.34677.log.gz 2024-09-21T23:14:11.532 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42427.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39463.log.gz 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35589.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39557.log.gz 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40257.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42427.log.gz 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35589.log.gz 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34672.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40257.log.gz 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34672.log.gz 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42974.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35812.log 2024-09-21T23:14:11.533 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43519.log: /var/log/ceph/ceph-client.admin.42974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42974.log.gz 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33044.log 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39604.log 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35812.log.gz 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33044.log.gz 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38157.log 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41101.log 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39604.log.gz/var/log/ceph/ceph-client.admin.38157.log: 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38157.log.gz 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41892.log 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43545.log 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41101.log.gz 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-09-21T23:14:11.534 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41892.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39073.log 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43545.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34670.log 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42021.log 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39073.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39012.log 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34670.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42021.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38013.log 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39012.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43155.log 2024-09-21T23:14:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43182.log 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38013.log: /var/log/ceph/ceph-client.admin.43155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38013.log.gz 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43155.log.gz 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37420.log 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38298.log 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43182.log.gz 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38857.log 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37420.log.gz 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38298.log.gz 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41559.log 2024-09-21T23:14:11.536 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38407.log 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38857.log: /var/log/ceph/ceph-client.admin.41559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38857.log.gz 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41559.log.gz 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42752.log 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39942.log 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38407.log.gz 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42752.log.gz 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33426.log 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39942.log.gz 2024-09-21T23:14:11.537 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38462.log 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33426.log.gz 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40121.log 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34015.log 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38462.log.gz 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40121.log.gz 2024-09-21T23:14:11.538 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32633.log 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39437.log 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34015.log.gz 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32633.log.gz 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32518.log 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38228.log 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39437.log: /var/log/ceph/ceph-client.admin.32518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39437.log.gz 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32518.log.gz 2024-09-21T23:14:11.539 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41644.log 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36237.log 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38228.log.gz 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41644.log.gz 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39387.log 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37354.log 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36237.log: /var/log/ceph/ceph-client.admin.39387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36237.log.gz 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39387.log.gz 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37080.log 2024-09-21T23:14:11.540 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38761.log 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37354.log.gz 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37080.log.gz 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38364.log 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32824.log 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38761.log.gz 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38364.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38364.log.gz -5 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43363.log 2024-09-21T23:14:11.541 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32824.log.gz 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34572.log 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40774.log 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43363.log.gz 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34267.log 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34572.log.gz 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33798.log 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40774.log.gz 2024-09-21T23:14:11.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42661.log 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34267.log.gz 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39760.log 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33798.log.gz 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41097.log 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42661.log.gz 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40176.log 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39760.log.gz 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41097.log.gz 2024-09-21T23:14:11.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37208.log 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40176.log.gz 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37857.log 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36152.log 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37208.log.gz 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32637.log 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37857.log.gz 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38124.log 2024-09-21T23:14:11.544 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36152.log.gz 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42635.log 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32637.log.gz 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43441.log 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38124.log.gz 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34513.log 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42635.log.gz 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43441.log.gz 2024-09-21T23:14:11.545 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34817.log 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/ceph-client.admin.34513.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26616.log 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34513.log.gz 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36397.log 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34817.log.gz 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26616.log.gz 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36870.log 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36059.log 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36397.log.gz 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42765.log 2024-09-21T23:14:11.546 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36870.log.gz 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36059.log.gz 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37898.log 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42765.log.gz 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41954.log 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43467.log 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37898.log.gz 2024-09-21T23:14:11.547 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41954.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41954.log.gz 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37052.log 0.0% 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43467.log.gz 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39682.log 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40055.log 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37052.log.gz 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33396.log 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39682.log.gz 2024-09-21T23:14:11.548 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33658.log 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40055.log: /var/log/ceph/ceph-client.admin.33396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40055.log.gz 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40341.log 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33396.log.gz 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35310.log 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33658.log: /var/log/ceph/ceph-client.admin.40341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41958.log 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40341.log.gz 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33658.log.gz 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35310.log.gz 2024-09-21T23:14:11.549 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40982.log 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39649.log 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41958.log: /var/log/ceph/ceph-client.admin.40982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39097.log 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41958.log.gz 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40982.log.gz 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39649.log.gz 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34863.log 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36765.log 2024-09-21T23:14:11.550 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39097.log: /var/log/ceph/ceph-client.admin.34863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35399.log 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39097.log.gz 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34863.log.gz 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36765.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36765.log.gz -5 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32852.log 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41724.log 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35399.log: /var/log/ceph/ceph-client.admin.32852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41773.log 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35399.log.gz 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32852.log.gz 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41724.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.41724.log.gz 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/ceph-client.admin.43935.log 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33318.log 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41773.log: /var/log/ceph/ceph-client.admin.43935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34930.log 2024-09-21T23:14:11.551 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41773.log.gz 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43935.log.gz 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33318.log.gz 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33893.log 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34930.log: /var/log/ceph/ceph-client.admin.40566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37774.log 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34930.log.gz 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37265.log 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33893.log.gz 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37774.log.gz 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36098.log 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37265.log: /var/log/ceph/ceph-client.admin.36211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40595.log 2024-09-21T23:14:11.552 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37265.log.gz 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36098.log.gz 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39225.log 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42375.log 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40595.log: /var/log/ceph/ceph-client.admin.39225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37610.log 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40595.log.gz 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39225.log.gz 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42375.log.gz 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40462.log 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35027.log 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37610.log: /var/log/ceph/ceph-client.admin.40462.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.37610.log.gz /var/log/ceph/ceph-client.admin.37237.log 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T23:14:11.553 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40462.log.gz 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35027.log.gz 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42583.log 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38958.log 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37237.log.gz 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39305.log 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42583.log.gz 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36300.log 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38958.log: /var/log/ceph/ceph-client.admin.39305.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39305.log.gz 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38958.log.gz 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36300.log.gz 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43179.log 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37424.log 2024-09-21T23:14:11.554 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.43179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33471.log 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43179.log.gz 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37424.log.gz 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32644.log 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32538.log 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33471.log.gz 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32644.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42479.log 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32644.log.gz 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32564.log 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32538.log.gz 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42212.log 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42479.log: /var/log/ceph/ceph-client.admin.32564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33056.log 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42479.log.gz 2024-09-21T23:14:11.555 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32564.log.gz 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35775.log 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42212.log.gz 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33056.log: /var/log/ceph/ceph-client.admin.35775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34301.log 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33056.log.gz 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35775.log.gz 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35260.log 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34424.log 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34301.log: /var/log/ceph/ceph-client.admin.35260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43162.log 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34301.log.gz 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35260.log.gz 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34424.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34424.log.gz -5 2024-09-21T23:14:11.556 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33619.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42136.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43162.log: /var/log/ceph/ceph-client.admin.33619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41749.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33619.log.gz 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43162.log.gz 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42136.log.gz 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32632.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41749.log: /var/log/ceph/ceph-client.admin.32632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33344.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41749.log.gz 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32632.log.gz 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38916.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33344.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34778.log 2024-09-21T23:14:11.557 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33344.log.gz 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35716.log 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38916.log.gz 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34778.log.gz 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41616.log 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42767.log 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35716.log.gz 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41616.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40009.log 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42767.log.gz 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38604.log 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42957.log 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40009.log.gz 2024-09-21T23:14:11.558 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38604.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26358.log 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38604.log.gz 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42957.log.gz 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34335.log 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37112.log 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26358.log: /var/log/ceph/ceph-client.admin.34335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26358.log.gz 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34335.log.gz 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34224.log 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38546.log 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37112.log: /var/log/ceph/ceph-client.admin.34224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37112.log.gz 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34224.log.gz 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41798.log 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38603.log 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38546.log: /var/log/ceph/ceph-client.admin.41798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38546.log.gz 2024-09-21T23:14:11.559 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41798.log.gz 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35920.log 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40866.log 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38603.log.gz 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35920.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35920.log.gz -5 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40866.log.gz 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35677.log 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36839.log 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.35677.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35677.log.gz 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38495.log 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32643.log 2024-09-21T23:14:11.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36839.log: /var/log/ceph/ceph-client.admin.38495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36839.log.gz 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38495.log.gz 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43988.log 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40707.log 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32643.log: /var/log/ceph/ceph-client.admin.43988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32643.log.gz 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43988.log.gz 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40707.log.gz 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43675.log 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34019.log 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37173.log: /var/log/ceph/ceph-client.admin.43675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43675.log.gz 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42763.log 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41149.log 2024-09-21T23:14:11.561 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34019.log: /var/log/ceph/ceph-client.admin.42763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34019.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42763.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38039.log 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41149.log: /var/log/ceph/ceph-client.admin.38039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41149.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38039.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33032.log 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42557.log 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34613.log: /var/log/ceph/ceph-client.admin.33032.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33032.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33717.log 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33058.log 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42557.log: /var/log/ceph/ceph-client.admin.33717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42557.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33717.log.gz 2024-09-21T23:14:11.562 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38769.log 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35221.log 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33058.log: /var/log/ceph/ceph-client.admin.38769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33058.log.gz 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38769.log.gz 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33830.log 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43571.log 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33830.log: /var/log/ceph/ceph-client.admin.35221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35221.log.gz 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33830.log.gz 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34117.log 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37293.log 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43571.log.gz 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34117.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32638.log 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37293.log.gz 2024-09-21T23:14:11.563 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42753.log 2024-09-21T23:14:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42713.log 2024-09-21T23:14:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32638.log.gz 2024-09-21T23:14:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42753.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43493.log 2024-09-21T23:14:11.564 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T23:14:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42713.log.gz 2024-09-21T23:14:11.565 INFO:teuthology.orchestra.run.smithi175.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21504.log 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41588.log 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43493.log: /var/log/ceph/ceph.tmp-client.admin.21504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43493.log.gz 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21504.log.gz 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26512.log 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33060.log 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41588.log: /var/log/ceph/ceph-client.admin.26512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41588.log.gz 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26512.log.gz 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38340.log 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41676.log 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33060.log: /var/log/ceph/ceph-client.admin.38340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33060.log.gz 2024-09-21T23:14:11.584 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38340.log.gz 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42505.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41676.log.gz 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42954.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40673.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42505.log.gz 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42954.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43597.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42954.log.gz 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40673.log.gz 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35055.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42687.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43597.log: /var/log/ceph/ceph-client.admin.35055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43597.log.gz 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35055.log.gz 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37671.log 2024-09-21T23:14:11.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42687.log.gz 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41321.log 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.37671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37671.log.gz 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35868.log 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36981.log 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41321.log.gz 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35868.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40202.log 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35868.log.gz 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36981.log: 28.2% -- replaced with /var/log/ceph/ceph-client.admin.36981.log.gz 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33059.log 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40202.log.gz 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43831.log 2024-09-21T23:14:11.586 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33059.log.gz 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36683.log 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34166.log 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43831.log: /var/log/ceph/ceph-client.admin.36683.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43831.log.gz 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36683.log.gz 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33370.log 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42274.log 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34166.log: /var/log/ceph/ceph-client.admin.33370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34166.log.gz 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33370.log.gz 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39194.log 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41672.log 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42274.log.gz 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39194.log.gz 2024-09-21T23:14:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32836.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41672.log.gz 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37329.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35159.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32836.log: /var/log/ceph/ceph-client.admin.37329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32836.log.gz 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37329.log.gz 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33947.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40232.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35159.log.gz 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33947.log.gz 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35508.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40232.log.gz 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42244.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40941.log 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35508.log: /var/log/ceph/ceph-client.admin.42244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35508.log.gz 2024-09-21T23:14:11.588 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42244.log.gz 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39855.log 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.40941.log: /var/log/ceph/ceph-client.admin.40539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40941.log.gz 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33240.log 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32493.log 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39855.log: /var/log/ceph/ceph-client.admin.33240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39855.log.gz 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33240.log.gz 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32635.log 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39915.log 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32493.log.gz 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32635.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32468.log 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T23:14:11.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.39915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39915.log.gz 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32833.log 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32468.log: /var/log/ceph/ceph-client.admin.32833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32468.log.gz 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32833.log.gz 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43779.log 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38411.log 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43779.log.gz 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38411.log.gz 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35373.log 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41259.log 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43183.log: /var/log/ceph/ceph-client.admin.35373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-21T23:14:11.590 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35373.log.gz 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43649.log 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32829.log 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.41259.log: /var/log/ceph/ceph-client.admin.43649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41259.log.gz 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43649.log.gz 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33266.log 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.32829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32829.log.gz 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33266.log.gz 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33946.log 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42349.log 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.33946.log: /var/log/ceph/ceph-client.admin.42349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33946.log.gz 2024-09-21T23:14:11.591 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42349.log.gz 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36502.log 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.43883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.42755.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35424.log 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.36502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36502.log.gz 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33042.log 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38719.log 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.35424.log: /var/log/ceph/ceph-client.admin.33042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35424.log.gz 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33042.log.gz 2024-09-21T23:14:11.592 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.38719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38719.log.gz 2024-09-21T23:14:11.646 INFO:teuthology.orchestra.run.smithi175.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T23:14:11.737 INFO:teuthology.orchestra.run.smithi150.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T23:14:11.770 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T23:14:12.358 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-21T23:14:12.800 INFO:teuthology.orchestra.run.smithi150.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T23:14:13.126 INFO:teuthology.orchestra.run.smithi175.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T23:14:13.278 INFO:teuthology.orchestra.run.smithi175.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T23:14:13.962 INFO:teuthology.orchestra.run.smithi150.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T23:14:13.971 INFO:teuthology.orchestra.run.smithi175.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T23:14:14.431 INFO:teuthology.orchestra.run.smithi150.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T23:14:14.483 INFO:teuthology.orchestra.run.smithi175.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T23:14:14.484 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-21T23:14:14.484 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m3.051s 2024-09-21T23:14:14.484 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m8.989s 2024-09-21T23:14:14.484 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.384s 2024-09-21T23:14:15.102 INFO:teuthology.orchestra.run.smithi150.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T23:14:15.103 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-21T23:14:15.103 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m3.712s 2024-09-21T23:14:15.103 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m10.825s 2024-09-21T23:14:15.103 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.558s 2024-09-21T23:14:15.104 INFO:tasks.ceph:Archiving logs... 2024-09-21T23:14:15.104 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/remote/smithi150/log 2024-09-21T23:14:15.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:14:16.456 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/remote/smithi175/log 2024-09-21T23:14:16.457 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:14:17.407 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T23:14:17.415 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T23:14:17.416 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:14:17.418 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:14:17.498 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T23:14:17.507 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T23:14:17.507 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:14:17.510 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:14:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:14:17.519 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2024-09-21T23:14:17.519 INFO:teuthology.orchestra.run.smithi150.stdout:+hv01.front.sepi 67.205.162.81 3 u 44 64 177 0.188 -0.659 0.698 2024-09-21T23:14:17.519 INFO:teuthology.orchestra.run.smithi150.stdout:+hv02.front.sepi 96.245.170.99 3 u 44 64 177 0.104 +2.081 0.772 2024-09-21T23:14:17.519 INFO:teuthology.orchestra.run.smithi150.stdout:*hv03.front.sepi 74.6.168.72 3 u 44 64 177 0.113 -2.127 0.797 2024-09-21T23:14:17.519 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:14:17.523 INFO:teuthology.orchestra.run.smithi175.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:14:17.523 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================== 2024-09-21T23:14:17.523 INFO:teuthology.orchestra.run.smithi175.stdout:*hv01.front.sepi 67.205.162.81 3 u 40 64 177 0.091 -0.007 2.710 2024-09-21T23:14:17.523 INFO:teuthology.orchestra.run.smithi175.stdout:+hv02.front.sepi 96.245.170.99 3 u 42 64 177 0.068 +2.665 2.705 2024-09-21T23:14:17.524 INFO:teuthology.orchestra.run.smithi175.stdout:+hv03.front.sepi 74.6.168.72 3 u 43 64 177 0.063 -3.241 1.561 2024-09-21T23:14:17.524 INFO:teuthology.orchestra.run.smithi175.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:14:17.524 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T23:14:17.533 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T23:14:17.533 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T23:14:17.541 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T23:14:17.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T23:14:17.557 INFO:teuthology.task.internal:Duration was 888.043212 seconds 2024-09-21T23:14:17.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T23:14:17.565 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T23:14:17.565 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:14:17.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:14:17.603 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T23:14:17.603 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-09-21T23:14:17.604 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:14:17.654 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T23:14:17.654 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:14:17.664 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T23:14:17.664 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:14:17.698 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:14:17.736 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T23:14:17.736 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:14:17.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:14:17.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T23:14:17.856 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T23:14:17.856 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:14:17.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:14:17.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T23:14:17.886 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:14:17.910 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:14:17.918 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-09-21T23:14:17.932 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-09-21T23:14:18.019 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:14:18.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:14:18.065 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:14:18.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:14:18.070 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T23:14:18.079 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T23:14:18.080 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/remote/smithi150 2024-09-21T23:14:18.080 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:14:18.134 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914506/remote/smithi175 2024-09-21T23:14:18.134 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:14:18.155 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T23:14:18.155 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:14:18.170 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:14:18.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T23:14:18.210 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T23:14:18.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T23:14:18.222 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T23:14:18.223 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:14:18.225 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:14:18.229 INFO:teuthology.orchestra.run.smithi150.stdout: 658636 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:14 /home/ubuntu/cephtest 2024-09-21T23:14:18.248 INFO:teuthology.orchestra.run.smithi175.stdout: 658466 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:14 /home/ubuntu/cephtest 2024-09-21T23:14:18.249 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T23:14:18.259 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T23:14:18.281 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} duration: 888.0432119369507 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T23:14:18.281 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T23:14:18.401 INFO:teuthology.run:pass